MySQL Limits

As a central component of the popular LAMP Stack, MySQL is a very versatile and popular database server. It can be easily configured to handle a relatively large number of users and is used by many high profile companies including Facebook, Youtube, Alcatel and many others. See https://www.mysql.com/customers/

One drawback, however is the default LAMP installation does not allow that many mysql connections or handle large databases well. The following configuration changes allows mysql to handle a fairly large database (500-800 tables) and user load (5000-10000 users).

These changes are suggested for RHEL / CentOS 7

  • Update my.cnf

nano /etc/my.cnf

max_connections = 500
key_buffer_size = 2048M
table_definition_cache = 4K
open_files_limit = 3K
query_cache_size = 128M
query_cache_limit = 128M
join_buffer_size = 4M
  • Update system limits on open files

To increase edit nano /etc/sysctl.conf add the below line, save and exit:

fs.file-max = 100000
  • Update mysql user limits on files and processes opened
Edit /etc/security/limits.conf add the below lines before the #End, save and exit:
mysql hard nofile 1024000
mysql soft nofile 1024000
  • Make the changes effective:
sudo systemctl daemon-reload
  • Restart mysql server
systemctl restart mysqld.service

Angular2 Chat Demo

This demo was created at the request of a company for a chat demonstration web app, suitable for desktops and mobile. This demo was inspired from https://github.com/start-angular/angular2-node-socket-io-chat-app.

If you paste the URL of a YouTube video, that video plays right in the chat window.

It also allows you to attach files and pictures. On mobile devices it allows you to send emoji. In the future plans are in place for an emoji selector on computers.

There is no login feature, it’s a public chat room, where participants only need to enter a nickname.

You can find the code for this demo on GitHub

A live demo is also available here.

Shop Problem

Here is the PHP Solution to an actual interview question:

Create a class called Shop. It should contain two methods:

buy($now, $quantity, $expiry)

and 

sell($now, $quantity)

Method buy accepts 3 parameters:

  • now – an integer that represents the present time
  • quantity – an integer for the amount currently being brought into the shop
  • expiry – an integer that represents the time at which this shipment expires

Method sell accepts 2 parameters:

  • now – an integer that represents the present time
  • quantity – an integer for the amount currently being sold from the store

This shop does not sell expired goods and tries to sell from the stock that is going to expire first. The task is to implement this class.

<?php

class Shop
{

  public $queue;
  public $apples;
  public $quantity;

  public function __construct()
  {
    $this->queue  = new SplMinHeap();
    $this->apples = array();
  }

  public function check_apples($now)
  {
    while ($this->queue->top() <= $now)
    {
      $this->delete_current_apple();
    }
  }
  
  private function debug($msg){
    return;
    echo "<div style=\"border:1px solid black;\"><pre>";
    var_dump($msg);
    echo "</pre></div>";
  }

  public function buy($now, $quantity, $expiry)
  {
    $this->queue->insert(intval($expiry));
    $this->apples[intval($expiry)] = intval($quantity);
    $this->quantity                += intval($quantity);
    $this->check_apples($now);
    return $this->quantity;
  }

  private function get_current_apple_quantity()
  {
    $cur = intval($this->queue->top());
    return intval($this->apples[$cur]);
  }

  private function delete_current_apple()
  {
    $cur            = intval($this->queue->top());
    $this->queue->extract();
    $this->quantity -= $this->apples[$cur];
    unset($this->apples[$cur]);
  }

  private function remove_apples($req_quantity)
  {
    $quantity = intval($req_quantity);
    $this->debug("Removing $quantity from " . $this->get_current_apple_quantity() . "<br />");
    $this->debug($this->quantity);
    if ($this->get_current_apple_quantity() > $quantity)
    {
      $cur                = $this->queue->top();
      $this->apples[$cur] -= $quantity;
      $this->quantity     -= $quantity;
    }
    $this->debug($this->quantity);
  }

  private function sell_helper($req_quantity)
  {
    $quantity     = intval($req_quantity);
    $cur_quantity = 0;
    do
    {
      $this->debug("Starting with " . $this->quantity);
      $cur_apple_count = $this->get_current_apple_quantity();
      $cur_quantity    += $cur_apple_count;
      $quantity_this_round = ($quantity >= $cur_quantity)?$cur_quantity:($cur_quantity - $quantity);

      if ($quantity_this_round >= $cur_apple_count)
      {
        $this->debug("deleting $quantity_this_round >= $cur_apple_count");
        $this->delete_current_apple();
      }
      else
      {
        if ($quantity_this_round < $cur_apple_count)
        {
          $this->remove_apples($cur_apple_count - $quantity_this_round);
        }
        else{
          $this->debug("unreachable code");
        }
        $cur_quantity = $quantity;
      }
      $this->debug($this->quantity);
      $this->debug("while ($cur_quantity < $quantity);<br />");
    }
    while ($cur_quantity < $quantity);
  }

  public function sell($now, $quantity)
  {
    $this->check_apples($now);
    $this->sell_helper($quantity);
    return $this->quantity;
  }

}

 

Simple Todo List Using Angular

This project took inspiration from this post.

It was enhanced by adding form validation and a few other features. As those features appear they will be documented here on this blog.

This application has AngularJS as the front end with NodeJS as the backend.

It is very simple and all the corresponding code can be found on GitHub.

Script to Incrementally Delete old Backups

This Script Assumes a folder with daily backups exists. It incrementally deletes backups. It is configurable from command line switches and comes with a default schedule already setup.

<?php

$backup_dir          = "/home/demo/backup";
$keep_files_days     = 14;
$extension_to_delete = "tgz";
$size_to_delete      = 0;
$test                = false;
$progressive         = true;
$log                 = true;
$date                = date("YmdHis");
$log_file            = "delete_old_backups_$date.log";
$log_path            = "/var/log/backup_scripts/";


$log_string = "---------------------------------------------------------\n"
  . "Starting Delete_old_backups script\nDate $date\n\n\n";

/**
 * @var Integer number of days to keep daily backups
 */
$daily   = 7;
$weekly  = 4;
$monthly = 12;

//check for command line argument
foreach ($argv as $arg)
{
  $item = explode("=", $arg);
  switch ($item[0])
  {
    case "dir":
      $backup_dir          = $item[1];
      break;
    case "keep":
      $keep_files_days     = $item[1];
      break;
    case "extension":
      $extension_to_delete = $item[1];
      break;
    case "size":
      $size_to_delete      = $item[1];
      break;
    case "progressive":
      $progressive         = $item[1];
      break;
    case "daily":
      $daily               = $item[1];
      break;
    case "weekly":
      $weekly              = $item[1];
      break;
    case "monthly":
      $monthly             = $item[1];
      break;
    case "log":
      $log                 = $item[1];
      break;
    case "log_file":
      $log_file            = $item[1];
      break;
    case "log_path":
      $log_path            = $item[1];
      break;
    case "help":
      echo get_help();
      break;
    case "test":
      $test                = ($item[1] === "true");
      break;
  }
}

/**
 * Setting up timestamps to compare daily, weekly, monthly
 */
$daily_days          = $daily;
$weekly_days         = $weekly * 7;
$monthly_days        = $monthly * 30;
$daily_file_string   = date("YmdHis", strtotime("-$daily_days day")) . ".tgz";
$weekly_file_string  = date("YmdHis", strtotime("-$weekly_days day")) . ".tgz";
$monthly_file_string = date("YmdHis", strtotime("-$monthly_days day")) . ".tgz";
$first_month         = true;
$first_week          = true;

$weekly_position  = $weekly - 1;
$monthly_position = $monthly - 1;

$weekly_array = array();
for ($i = ($weekly - 1); $i >= 0; $i--)
{
  $week_start_date  = ($i * 7);
  $week_end_date    = ($i * 7) - 7;
  $weekly_array[$i] = array(
    "keep"           => false,
    "start_filename" => date("YmdHis", strtotime("-$week_start_date day")) . ".tgz",
    "end_filename"   => date("YmdHis", strtotime("-$week_end_date day")) . ".tgz",
  );
}

$monthly_array = array();
for ($i = ($monthly - 1); $i >= 0; $i--)
{
  $month_start_date  = ($i * 30);
  $month_end_date    = ($i * 30) - 30;
  $monthly_array[$i] = array(
    "keep"           => false,
    "start_filename" => date("YmdHis", strtotime("-$month_start_date day")) . ".tgz",
    "end_filename"   => date("YmdHis", strtotime("-$month_end_date day")) . ".tgz",
  );
}

/**
 * Listing files in directory
 */
$timestamp_to_keep = strtotime("-$keep_files_days day");
$files             = scandir($backup_dir);
$date_to_keep      = date("Ymd", $timestamp_to_keep);

foreach ($files as $file)
{
  $log_string .= "found file $file\n";
  $to_delete  = false;
  $file_path  = $backup_dir . "/" . $file;
  if (endsWith($file, ".$extension_to_delete"))
  {

    if ($progressive)
    {
      $daily_cmp = strcmp($file, $daily_file_string);
      /**
       * Backups are taken daily so we do nothing for the files less than the daily threshold
       */
      if ($daily_cmp < 0)
      {
        $weekly_cmp = strcmp($file, $weekly_file_string);

        /**
         * Checking for weekly files
         */
        if ($weekly_cmp >= 0)
        {
          if (can_delete($weekly_array, $weekly_position, $file, $first_week))
          {
            $to_delete = true;
          }
        }
        else
        {
          if (can_delete($monthly_array, $monthly_position, $file, $first_month))
          {
            $to_delete = true;
          }
        }//end monthly (else greater than 4 weeks ago)
      }
    }//end if $progressive = true
    else
    {
      //  echo "ends with $extension_to_delete\n";
      $timestamp_cmp = strcmp($file, $date_to_keep);
      //echo $timestamp_cmp . "\n";
      if (($timestamp_cmp < 0) && ($progressive == false ))
      {
        $to_delete = true;
        if ($debug){
          echo "to_delete $file\n";
        }
        //echo $file_path . "\n";
      }//end check for time and progressive == false;
    }//end else  (progressive == false)

    if ($to_delete)
    {
      $log_string .= "Deleting $file\n";
      if (!$test)
      {
        unlink($file_path);
      }
    }//end if($to_delete)
    else
    {
      $log_string .= "Keeping $file\n";
    }//end else (!$to_delete)
  }//end check for timestamp
}//end  foreach file
//var_dump($test);
$ret = file_put_contents($log_path . $log_file, $log_string, FILE_APPEND);
//var_dump($ret);
exit();

/* * ********************************************************************************** */

function can_delete(&$periods, &$period_pointer, $current_file, &$first_period)
{
  if (in_period($periods, $period_pointer, $current_file, $first_period))
  {
    if ($periods[$period_pointer]["keep"] == false)
    {
      $periods[$period_pointer]["keep"] = $current_file;
      return false;
    }
    else
    {
      return true;
    }
  }
  return true;
}

function in_period(&$periods, &$period_pointer, $current_file, &$first_period)
{
  $start_cmp = strcmp($periods["$period_pointer"]["start_filename"], $current_file);
  $end_cmp   = strcmp($periods["$period_pointer"]["end_filename"], $current_file);
  while (($end_cmp < 0) && ($period_pointer >= 0))
  {// && ($first_period === true)) {
    $period_pointer--;
    $start_cmp = strcmp($periods["$period_pointer"]["start_filename"], $current_file);
    $end_cmp   = strcmp($periods["$period_pointer"]["end_filename"], $current_file);
  }
  $first_period = false;


  if (($start_cmp < 0) && ($end_cmp >= 0))
  {
    return true;
  }
  return false;
}

function endsWith($haystack, $needle)
{
  $length = strlen($needle);
  if ($length == 0)
  {
    return true;
  }
  return (substr($haystack, -$length) === $needle);
}

function get_help()
{
  return <<<help
  Script to delete old backups.

  Options:

  help -- show this message

    dir={dir} -- Change the location of the backups

    case "progressive":

    case "daily":

    case "weekly":

    case "monthly":

    case "log":

    case "log_file":
  
    case "log_path":
help;
}

 

Setting up moodle on Ubuntu 16.04

This post is to demonstrate how to setup Moodle 3.3 on Ubuntu 16.04:

Install aptitude package manager.

sudo apt-get install aptitude

Install Apache2

sudo aptitude install apache2

Install mysql

sudo apt-get install mysql-server

Secure Mysql installation

mysql_secure_installation

Install PHP

sudo apt-get install php libapache2-mod-php php-mcrypt php-mysql php-pear php-gd php-mbstring php-xml php-xmlrpc php-curl php-mbstring php-mcrypt php-soap php-intl

Install Pear Modules

sudo pear upgrade-all

sudo pear install MDB2-2.5.0b5

sudo pear install “channel://pear.php.net/MDB2_Driver_mysqli-1.5.0b4″

#Note that pear/mdb2#mysql is no longer available in PHP 7.0 since the mysql extension was removed in this version

sudo pear install “channel://pear.php.net/LiveUser-0.16.14″

sudo pear install HTML_Template_it

Install PIP (Python’s pear equivalent)

sudo apt-get install python-pip

sudo apt-get install unzip

sudo apt-get install elinks (text browser, to download moodle)

Install SSL:

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/ssl/private/apache-selfsigned.key -out /etc/ssl/certs/apache-selfsigned.crt/etc/apache2/sites-available/default-ssl.conf

sudo nano /etc/apache2/sites-available/default-ssl.conf

Add/Edit the following lines:

ServerAdmin your_email@example.com
ServerName server_domain_or_IP
SSLCertificateFile      /etc/ssl/certs/apache-selfsigned.crt
SSLCertificateKeyFile /etc/ssl/private/apache-selfsigned.key

Enable SSL and SSL VirtualHost:

sudo a2ensite default-ssl
sudo a2enmod ssl

Configuration test:
sudo apache2ctl configtest

Restart apache2 to enable SSL settings:

sudo systemctl restart apache2

Mysql commands:

create database moodle;

create user ‘webadmin’@’%’ identified by ‘{some password}’;

grant all on moodle.* to ‘webadmin’@’%’;

Download Moodle:

cd  /tmp/

elinks https://download.moodle.org/download.php/stable33/moodle-latest-33.tgz

navigate to and click on the download now link

tar xfz moodle-latest-33.tgz

Note:: Change moodle-latest-33.tgz to the correct file name, given on the web page -> https://download.moodle.org/releases/latest/

download.moodle.org
Install Moodle on your own server (requires a web server with PHP and a database) by downloading one of the following packages or obtaining Moodle via Git.

Move all moodle files to the wwwroot:sudo mv moodle/* /var/www/html/

 

Configure config.php

sudo cp config-dist.php config.php

sudo nano config.php

Set host and database configuration to match those set above.

 

Browse to your New Moodle Installation:

http://{your IP}/