Scale out your Magento application with 
ElasticBeanstalk
About us 
Cofounder & CTO - 
Bitbull S.r.l. @Bitbull_it 
bitbull-team 
on Github 
kesonno 
on Twitter 
Cofounder - 
Cofounder - 
Corley S.r.l. @CorleyCloud 
UpCloo LTD @UpCloo 
on Github 
on Twitter 
wdalmut 
walterdalmut 
- Internet Of Things! @ Turin [CFP] - 15 Nov 
- CloudComputing @ Turin [CFP ASAP] 
internetof.it 
www.cloudconf.it
ElasticBeanstalk 
Scale your app workers and web apps 
Queue Daemons 
Load balanced web applications 
Mainly for Service Oriented Architecture
Autoscaling applied to web applications
Autoscaling in practice 
Dynamically added and removed! 
Machine to Machine 
Closed-loop control system
Scale apps is not simple 
How to handle dev/testing/production? (dynamic env) 
How to install/update softwares? (dynamic env) 
How to handle user sessions? (more than one node) 
How to handle/tail logs? (dynamic env) 
How to monitor all instances (dynamic env) 
And more... (all things are moving!)
Run different environment per app 
Typically you run: Production, Testing, Development
How to deploy the app? 
ElasticBeanstalk updates app using Git on push 
ElasticBeanstalk updates app using a pre-packaged Zip file 
ElasticBeankstalk offers also API 
Git Push -> Jenkins Build ZIP package -> ElasticBeankstalk Deploy
Different app versions per environment 
Easy distributed app deploy/rollback and testing/production application management
Different configuration per environment 
Upgrade your environment and switch your production without downtime
Production Env Swap
My production environment
Add a new environment
Swap production environment 
ElasticBeanstalk swap env URLs in order to simplify the upgrade
Destroy your old environment 
COST-SAVING! 
it is easier create and destroy environments than upgrade them
Environment Management
Control your Application 
Configurations
Application variables 
All environment variables are ported to your application in $_SERVER 
You can pass everything like: Memcached and Mysql configurations etc.
Scaling made easy
We need more customizations! 
Create a folder in your project root with name .ebextensions and 
append your configuration files with extension .config 
ElasticBeanstalk will use them during the application provisioning
Cronjobs runs on the leader instance only 
.ebextensions/05_cron_jobs.config 
container_commands: 
01_magento_cron_job: 
command: "cat .ebextensions/magento_cron_job.txt > /etc/cron.d/magento_cron_job && chmod 644 /etc/cron.d/magento_cron_leader_only: true 
All configuration files are just simple YAML files
Monitor your environment 
You can monitor many metrics with CloudWatch 
UDP/IP CloudWatch agent on local machine: https://github.com/wdalmut/cloudwatch-agent
What we are missing? 
Logs!
Grab all active instances logs
But my application logs? 
commands: 
21_application_logs: 
command: echo "/var/app/current/var/log/*.log" > myapp.conf 
cwd: /opt/elasticbeanstalk/tasks/bundlelogs.d 
22_application_logs: 
command: echo "/var/app/current/var/log/*.log" > myapp.conf 
cwd: /opt/elasticbeanstalk/tasks/systemtaillogs.d 
23_application_logs: 
command: echo "/var/app/current/var/log/*.log" > myapp.conf 
cwd: /opt/elasticbeanstalk/tasks/taillogs.d 
24_application_logs: 
command: echo "/var/app/current/var/log/*.log" > myapp.conf 
cwd: /opt/elasticbeanstalk/tasks/publishlogs.d 
.ebextensions/06_prepare_logs.config
The Magento side
Use Composer for dependencies 
ElasticBeanstalk uses Composer in order to prepare your application 
So, we can use composer hooks in order to connect all env variables to 
our Magento configuration 
http://github.com/magento-hackathon/magento-composer-installer
Composer hooks 
{ 
/** other composer configs **/ 
"scripts": { 
"post‐update‐cmd": [ 
"CorleyDeployMagento::localConf" 
], 
"post‐install‐cmd": [ 
"CorleyDeployMagento::localConf" 
] 
}, 
}
<?php 
namespace CorleyDeploy; 
use ComposerScriptEvent; 
class Magento 
{ 
public static function localConf(Event $event) 
{ 
// Generate a local.xml based on environment variables 
} 
}
The golden rule: 
The filesystem is not reliable
Keep media in a shared location 
ok but, what could we use?
NFS? 
is a single point of failure 
is not at web scale
Amazon S3? 
does not work out-of-the-box 
maybe in future...
Database?
Why not? 
work out-of-the-box (and is dead simple to configure) 
read access to db only once for each instance... 
...may be reduced to only one using a CDN (CloudFront)
Cache and sessions 
All cache backend and session handlers except "File" are fine 
Best options are Redis and Memcached (both available via Amazon 
ElastiCache)
TO BE DONE 
sitemap.xml 
Exchange data for third-party integration (ERP, CRM etc.) 
Integration with services (eg. bank gateway) that require IPs whitelist 
Use a proxy 
IP reservation 
Store media on Amazon S3 
Use DynamoDB session handler
Thanks for listening 
Any question?

Scale your Magento app with Elastic Beanstalk

  • 1.
    Scale out yourMagento application with ElasticBeanstalk
  • 2.
    About us Cofounder& CTO - Bitbull S.r.l. @Bitbull_it bitbull-team on Github kesonno on Twitter Cofounder - Cofounder - Corley S.r.l. @CorleyCloud UpCloo LTD @UpCloo on Github on Twitter wdalmut walterdalmut - Internet Of Things! @ Turin [CFP] - 15 Nov - CloudComputing @ Turin [CFP ASAP] internetof.it www.cloudconf.it
  • 3.
    ElasticBeanstalk Scale yourapp workers and web apps Queue Daemons Load balanced web applications Mainly for Service Oriented Architecture
  • 4.
    Autoscaling applied toweb applications
  • 5.
    Autoscaling in practice Dynamically added and removed! Machine to Machine Closed-loop control system
  • 6.
    Scale apps isnot simple How to handle dev/testing/production? (dynamic env) How to install/update softwares? (dynamic env) How to handle user sessions? (more than one node) How to handle/tail logs? (dynamic env) How to monitor all instances (dynamic env) And more... (all things are moving!)
  • 7.
    Run different environmentper app Typically you run: Production, Testing, Development
  • 8.
    How to deploythe app? ElasticBeanstalk updates app using Git on push ElasticBeanstalk updates app using a pre-packaged Zip file ElasticBeankstalk offers also API Git Push -> Jenkins Build ZIP package -> ElasticBeankstalk Deploy
  • 9.
    Different app versionsper environment Easy distributed app deploy/rollback and testing/production application management
  • 10.
    Different configuration perenvironment Upgrade your environment and switch your production without downtime
  • 11.
  • 12.
  • 13.
    Add a newenvironment
  • 14.
    Swap production environment ElasticBeanstalk swap env URLs in order to simplify the upgrade
  • 15.
    Destroy your oldenvironment COST-SAVING! it is easier create and destroy environments than upgrade them
  • 16.
  • 17.
    Control your Application Configurations
  • 18.
    Application variables Allenvironment variables are ported to your application in $_SERVER You can pass everything like: Memcached and Mysql configurations etc.
  • 19.
  • 20.
    We need morecustomizations! Create a folder in your project root with name .ebextensions and append your configuration files with extension .config ElasticBeanstalk will use them during the application provisioning
  • 21.
    Cronjobs runs onthe leader instance only .ebextensions/05_cron_jobs.config container_commands: 01_magento_cron_job: command: "cat .ebextensions/magento_cron_job.txt > /etc/cron.d/magento_cron_job && chmod 644 /etc/cron.d/magento_cron_leader_only: true All configuration files are just simple YAML files
  • 22.
    Monitor your environment You can monitor many metrics with CloudWatch UDP/IP CloudWatch agent on local machine: https://github.com/wdalmut/cloudwatch-agent
  • 23.
    What we aremissing? Logs!
  • 24.
    Grab all activeinstances logs
  • 25.
    But my applicationlogs? commands: 21_application_logs: command: echo "/var/app/current/var/log/*.log" > myapp.conf cwd: /opt/elasticbeanstalk/tasks/bundlelogs.d 22_application_logs: command: echo "/var/app/current/var/log/*.log" > myapp.conf cwd: /opt/elasticbeanstalk/tasks/systemtaillogs.d 23_application_logs: command: echo "/var/app/current/var/log/*.log" > myapp.conf cwd: /opt/elasticbeanstalk/tasks/taillogs.d 24_application_logs: command: echo "/var/app/current/var/log/*.log" > myapp.conf cwd: /opt/elasticbeanstalk/tasks/publishlogs.d .ebextensions/06_prepare_logs.config
  • 26.
  • 27.
    Use Composer fordependencies ElasticBeanstalk uses Composer in order to prepare your application So, we can use composer hooks in order to connect all env variables to our Magento configuration http://github.com/magento-hackathon/magento-composer-installer
  • 28.
    Composer hooks { /** other composer configs **/ "scripts": { "post‐update‐cmd": [ "CorleyDeployMagento::localConf" ], "post‐install‐cmd": [ "CorleyDeployMagento::localConf" ] }, }
  • 29.
    <?php namespace CorleyDeploy; use ComposerScriptEvent; class Magento { public static function localConf(Event $event) { // Generate a local.xml based on environment variables } }
  • 30.
    The golden rule: The filesystem is not reliable
  • 31.
    Keep media ina shared location ok but, what could we use?
  • 32.
    NFS? is asingle point of failure is not at web scale
  • 33.
    Amazon S3? doesnot work out-of-the-box maybe in future...
  • 34.
  • 35.
    Why not? workout-of-the-box (and is dead simple to configure) read access to db only once for each instance... ...may be reduced to only one using a CDN (CloudFront)
  • 36.
    Cache and sessions All cache backend and session handlers except "File" are fine Best options are Redis and Memcached (both available via Amazon ElastiCache)
  • 37.
    TO BE DONE sitemap.xml Exchange data for third-party integration (ERP, CRM etc.) Integration with services (eg. bank gateway) that require IPs whitelist Use a proxy IP reservation Store media on Amazon S3 Use DynamoDB session handler
  • 38.
    Thanks for listening Any question?