Multi-database metastore Server
This project is maintained by infsci2711
The goal of this project is to keep information (source description in terms of virtual data integration approach) about the available data stores. The information should include all useful aspects, e.g. url, port, username, password, database name, what protocol to use to connect, what connection format is, any other information about the datastore. The information should be collected in two ways:
1. Ask user to input whatever they can/need to input.
2. User info that user input (e.g. connection info), you need to connect to the data store and collect more metadata. E.g. table names, their schemas, some data statistics, e.g. how much data there is, etc.
1. login to the server
If you have mac or you have some linux OS (e.g. Ubuntu) then you would need to use terminal to connect to the server. Simply open the terminal and type:
ssh YOURUSERNAME@IP
where YOURUSERNAME is your user name that you recieve in email and the IP is the IP address of the server.
Then enter your password (note no characters will be printed while you are entering your password) and you should be logged into the server.
2. Install the essentials:
sudo apt-get install maven
sudo apt-get install git
sudo apt-get install openjdk-7-jdk
sudo apt-get install mysql-server
sudo apt-get install nginx
3. Create folders:
cd /opt
sudo mkdir project
cd project
4. Clone projects from Github
sudo git clone https://github.com/infsci2711/MultiDBs-Utils.git
sudo git clone https://github.com/infsci2711/MultiDBs-MetaStore-Server
sudo git clone https://github.com/infsci2711/MultiDBs-MetaStore-WebClient
5. Create symlink to the client code in the project folder
cd /usr/share/nginx
sudo rm -R html
sudo ln -sv /opt/project/MultiDBs-MetaStore-WebClient html
6. Setup database
Run the startup.sh file under “MultiDBs-MetaStore-Server” directory
The startup.sh should contain the following codes:
#!/bin/bash
Replace [username] and [password] with your own username and password of your MySQL database, then do
mysql -u [username] –p[password] < metastoredb.sql
sh startup.sh
in terminal
7. build the project by maven install
To build the project, run mvn instal like this:
cd MultiDBs-Utils
mvn install
cd MultiDBs-MetaStore-Server
mvn install
8. Start the server:
cd /opt/project/MultiDBs-MetaStore-Server/MultiDBsMetaStoreServerAPI/target
nohup java -jar multidbsmetastoreserverapi-0.1-SNAPSHOT.jar > log.out 2> error.log < /dev/null &
1) To show all datasources ids with dbNames, the API is:
Method: GET
http://54.152.26.131:7654/datasources/ids
and the result should be the JSON array: [{id:..., dbName:""},...]
2) To get all tables names for a given datasource API is as:
Method: GET
http://54.152.26.131:7654/datasources/{id}/tables
the {id} is the parameter that is for given id of the datasource, you return the list of table names ["", "", ...]
3) Get list of columns for given datasource id and table name, the API is:
Method: GET
http://54.152.26.131:7654/datasources/{id}/{tableName}/columns
the {id} and {tableName} are the parameters the result should be JSON array of column names ["", "", ...]
4) Get just one datasource info by datasource id. The API is as:
Method: GET
http://54.152.26.131:7654/datasources/{id}
{id} is the parameter
JSON: { "id": 1, "IPAddress": "ip1", "port": 123, "DBname": "dbName1", "username": "username1", "password": "password1", "DBtype": "MySQL", "tables": [ { "tableName" : "test", "columns": [ { "columnName": "field" } ] } ] }
5) To register a new datasource the API is:
Method: PUT
http://54.152.26.131:7654/datasources/add
JSON: { "IPAddress": "ip1", "port": 123, "DBname": "dbName1", "username": "username1", "password": "password1", "DBtype": "MySQL"}
6) Get all datasources info. The API is as:
Method: GET
http://54.152.26.131:7654/datasources