MySQL Crashes on Amazon EC2 WebServer

I never had any issues with MySQL on an amazon ec2 instance until one client of mine had a website that kept crashing. I located the MySQL log file and kept seeing the following fatal error:

Fatal error: cannot allocate memory for the buffer pool…

 

There were other errors, but a fatal error is the one I wanted to look into first as it is the one stopping the server.

 

Here is how I solved this issue. I hope it helps someone with a similar problem:

 

First, it is always a good idea to back up your MySQL databases on the server in case something goes wrong.

 

Then I tried deleting any unused databases, tables and data to allow for more disk space as that seemed to be an issue. However, after doing so, I checked the memory available on the webserver by issuing a df command from the command line and found that there was clearly plenty of space available before and after deleting the data from the databases. Therefore this was not apparently my issue. Logic brought me to thinking that if there was plenty of disk space and there was still an error related to memory, there may be something wrong with the swap. I came to discover that the micro instance I was using didn’t have a swap space and that was cause for alarm because as far as I know, swap is needed to initialize the database engines.

Therefore, please follow the following link to my tutorial on how to add swap space to a Linux amazon ec2 instance if you are having similar issues. This should work on any Linux web server by the way:

Adding Swap Space to Linux Webserver

 

How to set up Cron Job on Amazon EC2 Cloud Servers

Here I will show you how to set up a cron job on an Amazon Linux server. We used the basic Linux 64bit ami but these directions should work for any Linux web server

How to set up Cron Job on Amazon EC2 Cloud Servers

We used the basic linux 64bit ami but these directions should work for any linux web server

open a command prompt

type: which php
returns: /usr/bin/php

using the reponse from which php above, add the proper shebang to your php script like so:

#!/usr/bin/php -q

the  -q parameter makes it so you don’t get an email every time cron runs.

continuing from command prompt:

type: crontab -e

then use vim editor to add the following line to crontab to run  your php script every minute:

* * * * * /usr/bin/php /home/ec2-user/crontest.php

or every hour:

0 * * * * …

the five cron positions are as follows:

minute         0-59
hour           0-23
day of month   0-31
month          1-12
day of week    0-7 (sun = 0 or 7)

start a php file named crontest.php and enter the following contents:

———————-code—————————-
#!/usr/bin/php -q
<?php
//script to test cron on adam’s cloud server:
$message = “If ur reading this then the cron job worked!!!!nn”;

$myFile = “testFile.txt”;
$fh = fopen($myFile, ‘w’) or die(“can’t open file”);

fwrite($fh, $message);
$stringData = “Yay for cloud cron! script was home/ec2-user/crontest.phpn”;
fwrite($fh, $stringData);
fclose($fh);
?>
——————————————————–

start another php file named testFile.php. It will just be a blank txt file with no content at all.

Then upload crontest.php and testFile.php to /home/ec2-user and change permissions of both to 755 or 777 if 755 doesn’t work but 777 can cause a security issue.

Important note: it seems that if you change the regular crontab using crontab -e call that they files have to be in the user’s directory that you are in when making the call which in this case is user ec2-user

To get crontab-e to work, use following comand to switch users and try again:

su ec2-user

to do:
try using sudo crontab -e and puttiing cron files in differnt folders to see if it will work..

How to Access Amazon EC2 Server with FileZilla

I have posted this tutorial on how to access an EC2 Amazon web server with FileZilla because several people seem to have trouble doing so and the process is really simple if you are shown how to do it properly. Follow the below steps closely and you will have access to your Amazon EC2 server using FileZilla in no time.

Connecting to Amazon server in EC2 with FileZilla FTP Client

  1. when an Amazon cloud server is created, the creator is given a security key which is in the format of a .pem file. They then will have to create a .ppk file from the .pem file. Luckily FileZilla will do this for you when you import the .pem so just know that it is doing so and you will be okay.
  2. Open FileZilla and go to “Edit” in the tool bar. Click on “Edit”, then “settings”. After that the settings window will open. In the left of the settings window, click on “sftp”. Then, in the right of the settings window, click on the “add key file…” button and select and import the .ppk or .pem file you were given.
  3. Click “Okay” to close the settings window.
  4. In the main menu of FileZilla, click on “FIle”, then “Site manager” to open the site manager dialog. Click the “new site” button to add a site and give your new connection a name you can remember.
  5. In the” host” field, enter the ip address of the Amazon EC2 you want to connect to. you can use the short or long form of the IP whichever you have access to. It will look something like one of the following:ec2-67-204-29-109.compute-1.amazonaws.com or 67.204.29.109
  6. In the “protocol” field, select “SFTP -ssh transfer protocol” from the drop down.
  7. For the “login type”, select “Normal”
  8. For “User” enter the username for your type of instance. For most of my servers, this is ec2-user, so if you have a Linux AMI like mine, enter “ec2-user”. For Ubuntu servers, enter “ubuntu” and for other servers, find out the username and enter it here.
  9. Do not enter a password, just click the “Connect” button next and you are done and the FTP connection should be made if you did everything correctly.

That’s all there is to it.

Amazon Cloud Server Security Rules

In this tutorial, I will go into detail about how to set up security rules for a Linux AMI in an Amazon EC2 web server instance.

When you are creating your instance, you will click on instances in the main nav and then click to add a new instance. In the step of this process that asks you to create or select a security group, you will want to create a special one according to your preferences in most cases unless one of the default ones happens to cover your needs which I doubt. Below, I will go over the rules for creating one for a Linux server with a MySQL database and later a Linux server with a PostgreSQL database. Both are for web servers of course.

Security rules for Linux MySQL Web server

Ports Protocol Source
21 tcp 0.0.0.0/0
22 tcp 0.0.0.0/0
80 tcp 0.0.0.0/0
443 tcp 0.0.0.0/0
49152-65535 tcp 0.0.0.0/0
0-65535 udp 0.0.0.0/0
-1 icmp 0.0.0.0/0

when creating your instance, you will be given the opportunity to enter the above rules in. Simply follow the table and create duplicate rules and your web server with Linux and MySQL will work just fine as mine does.

 

Security rules for Linux PostgreSQL web server

Ports Protocol Source postgresql
21 tcp 0.0.0.0/0
22 tcp 0.0.0.0/0
80 tcp 0.0.0.0/0
443 tcp 0.0.0.0/0
5432 tcp 0.0.0.0/0
5434 tcp 0.0.0.0/0
5499 tcp 0.0.0.0/0
49152-65535 tcp 0.0.0.0/0
0-65535 udp 0.0.0.0/0
-1 icmp 0.0.0.0/0

Regardless of which set up you choose, they are entered the same way when you create the instance from your Amazon Management Console. Most of you will probably be using a MySQL database, but as a web developer, I do have a couple clients that use a PostgreSQL database, so it is relevant information for some people.

 Tips for creating Security Groups for an Amazon EC2 Instance

There are some things you should be aware of when creating your security groups. I included the following tips so that beginning web developers do not face the same problems I did when I first started in the cloud:

  1. Add names to your security groups that are descriptive such as Linux_MySQL or Linux_PostgreSQL for the above two.
  2. when launching a new instance, you can reuse security groups, but it seems as if they do not show up in the manage console….well they do, but they will say “no rules”, but yet they work, so this is something I will follow up on in other posts as I learn more.

Setting up an Elastic IP with Amazon EC2 Webserver

In this tutorial, I will describe in detail how to set up an elastic IP address and associate it with an EC2 instance from the Amazon Management Console.

What is an elastic IP

An elastic IP is Amazon’s equivelient to a static IP. they are unique mostly in one respect. They are IP addresses that get associated with an individual’s Amazon account. They can therefore be set to point to any one of your web server instances in your Amazon account. this can be handy if you every want to point the IP address to another instance in case one goes down or you simply want to change instances for a domain name.

Steps to creating an elastic ip

  1. login to your amazon account at aws.amazon.com and make a note of the instance ID that you want to add an elastic IP address to.
  2. From the Amazon EC2 tab and click Elastic IPs.
  3. Click Allocate New Address and choose EC2 from the drop-down list. Click Yes, allocate and make a note of the new IP.
  4. Right-click the newly created Elastic IP and choose Associate or check the box in front of it and click associate address at the top of the page and select the instance you want to attach it to.
  5. Be sure to click yes associate to complete the process.
  6. restart your server. If you need assistance with how to restart an Amazon EC2 instance, read this article.

 

 

Flexy Template Engine failed to write to directory

This solution is for you If you see the following error using Flexy Templates:

HTML_Template_Flexy fatal error: HTML_Template_Flexy::failed to write to /dirname/etc…

Solution:

go into your ftp client and find the lib directory for your Flex engine. The file you want is called class.page.php. You have to find the compileDir and templateDir variables and set them all to the correct path. What worked for me was using the entire path instead of the path from the directory the file was in, so I used something like this for the compileDir:

/opt/bitnami/apache2/htdocs/flexy_compiled_templates

and something like this for the templateDir:

/opt/bitnami/apache2/htdocs/templates

what you need to use will depend on your directory structure, so figure it out by finding the two respective directories and figuring out their complete paths.

Fix Can Not Write to Directory in Bitnami EC2 Server

Okay, I hope this saves someone some headaches. I have a server on Amazon cloud. It is a Bitnami LAPP stack, but this solution would be same for any Bitnami server. I couldn’t write to files from an application in my website.

Solution:

Make sure that permissions are set to 777 for testing, then you can tighten them later. TO do this, run the following command from the Linux command line:

sudo chmod -R 777 /dir_path/dir_name

Then the folder you want writable has to be owned by the daemon user. This is where I got stumped for an hour or so. My directory was owned by the bitnami user which I thought was correct until I found a post somewhere online that said to change it to daemon. Run the following command at the Linux command prompt to change the owner of your directory to daemon:

sudo chown -R daemon /dir_path/dir_name

FYI, the -R parameter causes both above commands to go into files and subdirectories and change permissions and ownership, so if that’s now what you want, simply exclude the -R in either or both of the above commands.

How to Clone an EC2 Instance

In this brief tutorial, I will describe how to clone an Amazon ec2 server instance.

  1. Log into your AWS account HERE.
  2. click to view your running instances and place a check by the instance you want to clone.
  3. Click the actions tab above the instance area and select “Create Image”.
  4. In the left navigation area, click “AMIs”.
  5. Right click on the AMI you just created and select “Launch Instance”

Then as you follow the prompts to launch the new instance you can make the necessary changes you may have left out the first time. For example, I found this to be the easiest way to add a new security group or open a port on an existing server.

Starting new server

after you have done the above, you will need to start the server and database like this after logging into the server with putty:

sudo service httpd start

sudo /etc/init.d/mysqld start

TIPS

If you want to use a security group from another server, the rues may not show up, but it did work when I did it. Don’t know why, at first I thought it had to be in same zone, but changed to same zone and still didn’t show up, but when I tested the cloned instance, it worked, so the security rules must be in place…

Restart HTTP Server from Command Line

How to restart web server from command line

Here is how to restart various servers from Linux command prompt.

Restarting a Linux Bitnami Lappstack apache server

sudo /opt/bitnami/ctlscript.sh restart apache

Restart a postgresql database on Bitnami LAPPstack

sudo /opt/bitnami/ctlscript.sh restart postgresql

Restarting other Amazon Linux APIs

To start most Ubuntu type servers and other Linux APIs use the following command to restart the server:

sudo service httpd restart

Then the following command to restart the database if you are using MySQL:

sudo service mysqld restart

 

Reset Your IP Address

How to get a new IP address

this method is one I find necessary quite a bit as a developer. You will find that often servers will block access because you either accessed too many times, downloaded too much data or entered an incorrect password to name a few things that get you blocked from misc servers. Here are the steps you can take on your desktop machine to reset your IP and regain access to those blocked servers you have been banned from.

Basic Steps

First, here are the basic high level steps for those in the know. If this doesn’t explain enough, read more detailed instructions in the next section”

1. Release your public IP
2. Pull the power from the modem
3. Change the MAC address of the device that had the public IP.
4. Power on the modem
5. Check your public IP.

Detailed Steps

Here are the more detailed steps that worked for me:

  1. Get your current IP address first by going to Google’s search engine and typeing in “whats my ip” and it will show you your current IP access that you are accessing the internet from. Write it down so you can confirm the change when you are done with these steps.
  2. In Windows, click Start and type in command and press enter to get a command prompt window.
  3. At the command prompt, type ifconfig/release.
  4. Unplug the cable modem.
  5. Change the mac address on router by going to http://192.168.1.1/ and basic settings. Get the mac address off of the back of the wireless router. if that one is used already, click to use the computer’s mac address.
  6. click apply or save to commit the changes and allow your wireless router to reset itself to apply changes.
  7. From the command prompt again, type ipconfig/release, wait 60 seconds and type ipconfig/renew just to be safe.
  8. Plugin the cable router and check your IP and confirm it has changed. If it didn’t work, try releasing and renewing your IP a couple more times and then repeat the entire process if it still doesn’t work, allowing more time between steps.

 

Renaming a Postgresql Database

How to rename a postgresql database

Gain access via Linux command line. I use Putty.

Get int psql by issuing the command without naming a database like this:

psql

Then issue the following command and press enter:

ALTER DATABASE orig_name RENAME TO new_name;

change orig_name to your original database name and new_name to the new database name you want to rename it to.

Check that it was successful by typing this at psql prompt:

l

and it will give you something like the following output:

Screenshot
Screenshot after renaming maximize_master to maximize_master1

 

Installing PHP and Mysql Database on Amazon EC2 Cloud Server

Installing PHP and Mysql from Linux command line

Here are the command line commands to get PHP and Mysql onto a Linux server without any software installed yet but basics. I did this on an Amazon Linux AMI:

TIP: Commands are in purple

sudo yum install php-mysql php php-xml php-mcrypt php-mbstring php-cli mysql httpd

sudo yum install mysql-server

sudo /etc/init.d/mysqld start

mysqladmin command to change root password:
mysqladmin -u root password admin

mysql -u root -p

type in password of “admin”

CREATE DATABASE MYSQL;

use MYSQL

CREATE USER ‘MYSQL_admin’@’localhost’ IDENTIFIED BY ‘mypass’;

GRANT ALL PRIVILEGES ON MYSQL TO ‘MYSQL_admin’@’localhost’;

CREATE USER ‘MYSQL_user’@’localhost’ IDENTIFIED BY ‘mypass’;

GRANT SELECT, INSERT, UPDATE, DELETE ON CREATEMYSQL.* TO ‘MYSQL_user’@’localhost’;
exit;

sudo chown ec2-user /var/www/html

sudo vim /var/www/html/index.php

Now you should have a server set up with mysql

Test by opening VI editor and making a test index.html or .php page here is how to open vi and start a new file:

sudo chown e2c-user /var/www/html

sudo vim /var/www/html/index.php

press i and insert some test text then hit esc and type :wq to save the file.

Now start the httpd server using the following command in Putty:
sudo service httpd start

if you got a green “ok” then you are good, if not go over the last several steps and find out what you did wrong.

Go to your Amazon admin console and click on instances to view your instances. Put a check next to the one you just made and copy the url that looks like this:
ec2-54-242-236-133.compute-1.amazonaws.com
you can shorten the url by just using the ip that starts after the ec2- and ends at the first period. Just replace the dashes with periods like this:
54.242.236.133
That is the short URL or IP you can use to access your web server. Type or copy and paste that into your web browser’s address bar and see if you can see your test text you entered with the VI editor a moment ago. If so, you are doing great.

Install phpmyadmin

Next, you can follow the below instructions if you want to install the popular phpmyadmin databaes GUI:

From the server command prompt, type:

cd /var/www/html
wget http://www.sourceforge.net/projects/phpmyadmin/files/phpMyAdmin/3.4.1/phpMyAdmin-3.4.1-all-languages.tar.bz2

tar -jxf phpMyAdmin-3.4.1-all-languages.tar.bz2 -C /var/www/html

mv phpMyAdmin-3.4.1-all-languages phpmyadmin

cd /var/www/html/phpmyadmin

mkdir config

cp /var/www/html/phpmyadmin/config.sample.inc.php /var/www/html/phpmyadmin/config/config.inc.php

sudo vim /var/www/html/phpmyadmin/config/config.inc.php

change  blowfish_secret to any other word and change allownopassword to true

go to: your server’s URL that looks something like http://33.234.51.182/phpmyadmin/
and enter user of root and the root password or one of the other users and passwords you created in this exercise. If you get the phpmyadmin interface, you did good.

 

 

Increasing Disc Space of Amazon EC2 Instance

Here are the steps to increasing disc space on an amazon ec2 instance.

NOTE: This tutorial has been updated to reflect recent changes on December 24th, 2016 because the original was written three years earlier. So the guide is up-to-date as of December, 2016 now.

  1. Login to your Amazon account’s management console HERE. Then go to your instances and stop the instance you want to increase the disc space on.
  2. Get the instance id of the server you want to increase disc space on. The instance id will look like this: i-5cdfdd2
  3. In the description tab for the selected instance, find the Volume ID that looks like: vol-00881476. Make a note of that too.
  4. In the same description tab, find the zone which looks like eu-west-1b and make a note of it as well.
  5. Select Volumes in the left navigation pane of the management console.
  6. Locate the Volume ID from step 3 and right click it then select “create snapshot”. Give it a name you will remember later and make a note of it.
  7. Click on “snapshots” in the main left navigation area and select “Create Volume” by right clicking on your new snapshot.
  8. Select the availability zone from step 4 and enter a new disc size. I made mine 110gb since I had a small instance with 160gb available. Check the aws docs to see the max allowed for your instance type.
  9. Click on “Volumes” again in the left column and find your old volume and detach it. When I tried this again in December of 2016, I got an error saying ti cannot detach root volume. I searched Amazon’s rather scattered documentation and found this note:[ NOTE: If an Amazon EBS volume is the root device of an instance, it cannot be detached while the instance is in the ‘running’ state. To detach the root volume, stop the instance first. Also, make sure to unmount any file systems on the device within your operating system before detaching the volume. Failure to unmount file systems, or otherwise properly release the device from use, can result in lost data and will corrupt the file system.] Now I did it without unmounting the file system because I knew this particular filesystem was not currently being used by anyone, but if it is a live site and is active, you definitely SHOULD unmount the filesystems first. All I did was clicked on “instances” in the left navigation, right clicked the running instance and selected “state/stop”.  After that wait for your instance to stop completely before continuing. You’ll see the icon spinning in front of your instance while it is stopping… After it is stopped you may go back to volumes and detach the volume successfully this time. Wait for it to detach…
  10. Find your new volume. Right click the volume and select “Attache Volume” then select the instance ID you noted in step 2. Enter sda1 after last slash. write down the volume id such as vol-0764a479.
  11. Now restart your instance, log in to it from putty or other command line interface and issue the command df -h and you will see your new volume information if everything worked. If disk space is the same continue reading, otherwise you are done.

When you run the df command from the command prompt and still see same disk space like in the following image, you need to increase the volume size from command prompt as I will instruct you how to do below:

df1

as you can see in the above image, i still had 8256952 one time I tried this when I was increasing a micro instance volume size from 8 gib to 16 gib, so I ran the following command from the command prompt:

resize2fs /dev/xvda1

then I ran typed df from command prompt again and got the following results which indicate the larger volume size of 16gib:

df2

Notice the size is now 16513960. By the way, if you run df -h instead of just df at the command prompt, it would say 16gb instead of 16513960. The h parameter puts it in human readable format.

That’s it. That is exactly how I increased my disc space on an Amazon ec2 instance recently. This was a linux instance, but the only difference for a Windows instance would be to replace sda1 with xdvf or something similar in step ten above. Good luck!