Preinstallations

At this page are some required steps listed. They are needed to compute STARQL queries with PostgreSQL or Apache Spark. You can use these installation steps to set up only one node (master) or optional for a whole cluster. You need a running instance of Ubuntu at every node you want to use.

At the picture above you can see a example of a computer cluster which consists of several nodes. One master node and several slaves (slave00, slave01, ...). The master node is connected over ssh with the slave nodes. In the instructions are the names used as in the picture. Your nodes may have another names! You can see the names under `/etc/hosts`.

1. Update
   Update all your nodes with:

   user@master:~$ sudo apt-get update && sudo apt-get -y dist-upgrade

   Then reboot:

   user@master:~$ sudo reboot

2. Maven
   You need to install maven to build the project from the repository and git to clone it:

   user@master:~$ sudo apt-get install maven git

3. Oracle Java 8
   To run the application you need to install Java 8 at every node. You can verify a Java installation with java -version. You should get something like this:

    java version "1.8.0_121"
    Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
    Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
   

   If not, then you do not have Java 8 installed. You can do it with the following commands:

   user@master:~$ sudo add-apt-repository ppa:webupd8team/java
   user@master:~$ sudo apt-get update
   user@master:~$ sudo apt-get install oracle-java8-installer

4. Hosts File
   In the hosts file at any node under /etc/hosts you should have at least:

   127.0.0.1        localhost
   
   # any name your node actually has
   127.0.1.1        name
   
   # your master node
   192.168.1.100    master
   
   # your slaves, when you use more than one node
   192.168.1.101    slave00
   192.168.1.102    slave01
   ...

5. User To avoid security issues you should add a new user and a new group at every node you want to use:

   user@master:~$ sudo addgroup cluster
   user@master:~$ sudo adduser --ingroup cluster starql

6. Configuring SSH
   You need a openssh-server running at every node. Even if you only use one node. You can do it with the command:

   user@master:~$ sudo apt-get install -y openssh-server

   Login as the new user with sudo su starql. Then you can generate a ssh key pair with the following command:

   starql@master:~$ ssh-keygen -t rsa -b 4096

   At the master node you need to add the generated public key to the list of authorized keys with the following command:

   starql@master:~$ ssh-copy-id -i ~/.ssh/id_rsa.pub localhost

   You can test it with the command:

   starql@master:~$ ssh localhost

   If you want to use more than one node, you need to copy the public key to the list of authorized keys at every slave node. You can do it with the following commands:

   starql@master:~$ ssh-copy-id -i ~/.ssh/id_rsa.pub slave00
   starql@master:~$ ssh-copy-id -i ~/.ssh/id_rsa.pub slave01
   ...

7. Disable IPv6
   We need to disable IPv6 at every node, because hadoop does not support it. To do it you need to add the following lines to /etc/sysctl.conf.

   # disable ipv6
   net.ipv6.conf.all.disable_ipv6 = 1
   net.ipv6.conf.default.disable_ipv6 = 1
   net.ipv6.conf.lo.disable_ipv6 = 1

8. PostgreSQL installation
   You need a running instance of PostgeSQL at the master node. The Spark back end also uses a PostgreSQL database as a data input. If you do not have installed PostgreSQL yet, then you can do it with the following command:

   user@master:~$ sudo apt-get install postgresql

   Modify /etc/postgresql/9.5/main/pg_hba.conf and change the line

   local   all             all                                     peer
   

   to

   local   all             all                                     md5
   

   Make sure you have these lines in the file pg_hba.conf:

   host    all             all             127.0.0.1/32            md5
   host    all             all             192.168.0.0/16          md5
   

   Modify /etc/postgresql/9.5/main/postgresql.conf and change the line

   #listen_addresses = 'localhost'`
   

   to

   listen_addresses = '*'`
   

   Restart PostgreSQL:

   user@master:~$ sudo service postgresql restart

   Change your password:

   user@master:~$ sudo -u postgres psql
   postgres=# ALTER USER Postgres WITH PASSWORD '<newpassword>';
   postgres=# \q
   

9. Reboot
   Reboot all your nodes:

   user@master:~$ sudo reboot

And Now?
Goto Home and continue with the next steps.