In this post we we will see how we deploy Phoenix application using AWS CodeBuild, CodeDeploy and CodePipeline
Create a Phoenix app to deploy
$ mix phx.new memories
$ mix ecto.create Compiling 13 files (.ex) Generated memories app The database for Notes.Repo has already been created
$ mix phx.gen.html Notes Note notes note:string * creating lib/memories_web/controllers/note_controller.ex * creating lib/memories_web/templates/note/edit.html.eex * creating lib/memories_web/templates/note/form.html.eex * creating lib/memories_web/templates/note/index.html.eex * creating lib/memories_web/templates/note/new.html.eex * creating lib/memories_web/templates/note/show.html.eex * creating lib/memories_web/views/note_view.ex * creating test/memories_web/controllers/note_controller_test.exs * creating lib/memories/notes/note.ex * creating priv/repo/migrations/20200415160343_create_notes.exs * creating lib/memories/notes.ex * injecting lib/memories/notes.ex * creating test/memories/notes_test.exs * injecting test/memories/notes_test.exs Add the resource to your browser scope in lib/memories_web/router.ex: resources "/notes", NoteController Remember to update your repository by running migrations: $ mix ecto.migrate
/notes resources to router.ex
scope "/", MemoriesWeb do pipe_through :browser get "/", PageController, :index resources "/notes", NoteController end
Run mix ecto.migrate and then your newly generated migration will run.
$ mix ecto.migrate 21:42:41.961 [info] == Running 20200415160343 Memories.Repo.Migrations.CreateNotes.change/0 forward 21:42:41.964 [info] create table notes 21:42:41.972 [info] == Migrated 20200415160343 in 0.0s
Now when you start
mix phx.server, you should see the app running at http://localhost:4000/notes
Make it ready for mix release
Uncomment following line in the
Now initialize git repo with
git init command and push it to GitHub.
In the next steps we will use AWS services for setting up CodeBuild, CodeDeploy and CodePipeline.
Create IAM User
We will create IAM User
We have created a username
memories-user and set a custom password to access AWS Management Console access.
Next under permissions, Choose attach existing policies directly.
AmazonEC2FullAccess AWSCodeDeployFullAccess AWSCodePipelineFullAccess AWSCodeBuildAdminAccess IAMFullAccess AmazonSNSReadOnlyAccess
After done, you will get a AWS console login for the new user. Go ahead and login with that new user. We will be only using that new account. You will be prompted to change your password on your first login. You should see your AWS Console dashboard.
AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy.
Let's create a new CodeBuild project. In your AWS console, you can find CodeBuild under the services tab in the header section or you can directly go to CodeBuild using this link. Go create a new project and you should see a form like this.
For Project name, I am using
memories-build. Under source, you need to select GitHub as Source Provider. You'll need to connect your GitHub account to AWS CodeBuild using OAuth. After connecting it, select 'Repository in my GitHub account'. Select the repository we created earlier.
Now, under 'Environment', we can go with 'Managed' image and select 'Ubuntu' as our operating system and 'Standard' as our runtime. We will leave rest of the section of Environment and Buildspec as it is.
Select 'Amazon S3' for Artifacts type. You can either create a new bucket that will store our project builds here first or select an existing bucket. For Artifacts packaging, select Zip and then click on 'Create build project'.
You should now see your build project under 'Build projects'
We have setup build process and now we want AWS to build our project on every push to GitHub. We can do that with AWS CodePipeline
AWS CodePipeline is a fully managed continuous delivery service that helps you automate your release pipelines for fast and reliable application and infrastructure updates. CodePipeline automates the build, test, and deploy phases of your release process every time there is a code change, based on the release model you define
Just like we created a CodeBuild project, we need to create a new CodePipeline project. You can find CodePipeline from 'Services' header in the menu or use this link. Let's go ahead and create new Pipeline using this form.
We'll use 'memories-pipeline' as the name and create a new service role. The name of it auto generated so let's stick to it. For Artifact store, you can select Default location or select different bucket to store artifacts from CodePipeline
Next step, let's add source stage. We are pushing our code to GitHub and want to build the source from it directly so we'll select GitHub as our provider. We need to connect CodePipeline to your GitHub account just like we did for CodeBuild. We then need to select GitHub project and the branch we want CodePipeline to watch. For change detection, we'll use GitHub webhooks.
After source stage, we will get to build stage. We just want to select Build provider as 'AWS CodeBuild' and select the build project that we had built earlier and click 'Next'
We now get to Deploy stage which we will be setting up later, so you can go ahead and skip this stage and then confirm all the changes to finish creating the CodePipeline project. You should see your pipeline that you created under 'Pipelines' section. If you go to the pipeline we just created, you should see that it starts fetching the code from GitHub repository and tries to build it and it fails. It is because we have not added our
buildspec.yml file to the repository yet.
Let's go ahead and add
buildspec.yml file to root of Phoenix project repo. Don't push to remote yet. We will need to do following steps first.
For our BuildProject, we need two environment variables to set. First is
SECRET_KEY_BASE and second is
DATABASE_URL With following steps we will get those.
SECRET_KEY_BASE, run following command on your machine
$ mix phx.gen.secret /4pTqhKeD87SO1+/6NlblLrB6HngJq9AinspgNLNKQXHO3nv5xlVbmZwDYWoaGuX
We will use the generated result as our
SECRET_KEY_BASE. Keep that with you.
Note = For our current memories-user IAM user doesn't have permision to create it, so make sure you log out and login with your main AWS root credentials or main credentials
Go to RDS and create a Postgres database instance.
Search for 'RDS' in the Services section and then choose to create new PostgreSQL database.
Once you create database, make sure you create database named
$ psql -h memories-db.xyzxyz.us-east-1.rds.amazonaws.com -U postgres -W Password: <enter your postgres password> psql (12.1, server 11.5) SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off) Type "help" for help. postgres=> CREATE DATABASE memories; CREATE DATABASE
DATABASE_URL should be looking something like below
We need to set two environment variables in the Environment for the CodeBuid project. Go and edit the the CodeBuild project. Choose 'Environment'
Now set the
DATABASE_URL in the CodeBuild for our build project's environment variable as below.
Also now push our
buildspec.yml from local Git repo to GitHub remote master branch.
Once you do that, you should see statues being changed to 'In progress' and then to 'Succeeded' along with the name of the commit and commit id for both 'Source' and 'Build' stages.
Creating EC2 instance
Now for running our Phoenix application, we need a EC2 instance. Search "EC2" in the Services and launch a new EC2 instance. Select "Ubuntu Server 18.04 LTS (HVM)", next, you can choose any instance type. We will use t2.micro instance for this article.
Under the Configure instance, we need to give IAM role for the instance.
We don't have the role needed for using CodeDeploy to deploy the project to our instance. So let's go ahead and create a new role
Select EC2 and click next
Attach following policies
AmazonEC2RoleforSSM AmazonEC2RoleforAWSCodeDeploy AmazonS3ReadOnlyAccess AmazonSSMReadOnlyAccess CloudWatchAgentServerPolicy
I give it a name as 'memories-codedeploy-ec2'
Now come back and refresh the IAM Role, you will see our new role in the list, choose it.
Now let's add a tag with Key as
server-name and value as
In this article, we will be running our Phoenix app on port
4000, so create new security group with type Custom TCP and port 4000. In the source, allow it from Anywhere
Next download KeyPair and we will need to SSH to instance.
Setup CodeDeploy Agent on EC2 instance
We need to set CodeDeploy Agent on our EC2 instance. SSH to the instance and run following command
sudo apt-get update sudo apt-get install -y ruby sudo apt-get install wget cd /home/ubuntu wget https://aws-codedeploy-us-east-1.s3.amazonaws.com/latest/install chmod +x ./install sudo ./install auto
In above commands running wget command with proper AWS region is important, that's why we use
aws-codedeploy-us-east-1 in the URL.
AWS CodeDeploy is a fully managed deployment service that automates software deployments to a variety of compute services such as Amazon EC2, AWS Fargate, AWS Lambda, and your on-premises servers.
Find out CodeDeploy link from Services header. Let's follow the same procedure to create new CodeDeploy as we did with CodeBuild and CodePipeline. We'll create a new CodeDeploy project. We need to create a new role. We'll select 'CodeDeploy' and under the use case. We will select 'CodeDeploy'
We don't need to change any permission policies this time and can simply proceed to last step where we need to add name for the role. We can call it 'memories-codedeploy'
Now we have our role ready, we can move ahead with creating a new CodeDeploy applicaiton. Click 'Create Application' in CodeDeploy, you should see a form like one below
We'll go ahead and give it a name
memories-codedeploy. We'll select EC2/On-premises for our Compute platform. After we create it, you'll notice we have a message saying 'In order to create a new deployment, you must create a deployment group'. So let's click on 'Create Deployment Group' at the bottom of the page.
We can call our deployment group 'memories-deployment-group' and select role that we just created. Our deployment type will be 'In place'
We will select 'Amazon EC2 instances' as our environment configuration. In order for CodeDeploy to find our EC2 instance, we had added a tag while creating the instance. We'll simply add the same key and value here.
In this article, we are not using any Load balancer so uncheck the box.
Just like we added
buildspec.yml, for CodeDeploy, we need
appspec.yml file for the CodeDeploy. It will contain instructions to carry during our deploy process. Let's create the file
appspec.yml in our project's root folder where we had added
As in above yaml file, we have a script to start the application. Create a folder named scripts in the root of your project and then add the following file named
Make sure change the permission of the file to be executable script.
$ chmod +x scripts/start.sh
We need to change
buildspec.yml as well, add following lines at the end of build: commands section in buildspec yaml
- cp appspec.yml _build/prod/rel/memories/ - cp -R scripts _build/prod/rel/memories/
This is to copy our appspec.yml and start script to final release so that we can use it while starting up the application.
Now commit newly added files to Git and push it to GitHub
Add Deploy Stage
Now let's go back to CodePipeline application. Right now we only have 'Source' and 'Build' stage setup. Let's add a 'Deploy` stage. Click 'Edit' at the top of the page.
On the edit page, there is 'Add stage' button after each stage. Let's click one right after the 'Build stage'. Let's give a name of our new stage as 'Deploy'. Once we have our 'Deploy' stage, we need to add action group.
We call our action as 'Deploy' and select 'AWS CodeDeploy' as our Action Provider. We need to select our CodeDeploy application and deployment group that we had created. Choose 'BuildArtifact' as our input artifacts. Click 'Save' button. And we have finally completed our setup.
Run database migration
Our Phoenix application is a basic CRUD application which talks with the database to store the notes as memories. We have one database migration in our application. To run the migration on our EC2 deployed application, create a file named `release.ex` in lib/memories folder as
We are using above as per https://hexdocs.pm/phoenix/releases.html
Now commit the code and push it to GitHub
In the CodePipeline project, we should see something as following now.
Now ssh to the EC2 instance and run following command from bin folder of the release
sudo ./memories eval "Memories.Release.migrate"
You should see something as below.
Now go and visit the our EC2 instasnce public IP URL
You should be able to create new notes.
Our application logs can be see on EC2 instance with following command
This article is inspired from original article - https://botsplash.com/blog/using-aws-code-suite-to-automate-build-and-deploy-a-simple-expressjs-project-86fb359c0358.html I have ported it for Phoenix application.