McNair Center IT

Server Build notes:

• RDP Documentation (IP Address: 128.42.44.182)
• Web Server Documentation (IP Address: 128.42.44.180)
• Database Server Documentation (IP Address: 128.42.44.181)
• Test Web Server Documentation (IP Address: 128.42.44.22)
• Power Backup

Other useful resources:

• Posgres Server Configuration
• Working with PostgreSQL
• Haas PhD Server Configuration
• Software Repository
• Patent Data Processing - SQL Steps

Wishlist:

• Server Backup Policy : under development
• McNair Security Policy
• Progress tracking platform
• Server maintenance FAQ
• NLP Infrastructure
• RDP Compute Performance

Semantic Form Data structure

Logins

• The initials for the SDC platinum on Father (the RDP server) are: mc

FAQ

In order to access the McNair Center resources, you generally need to be on the Rice network. Exceptions to this include accessing the dbase and webserver by ssh, and accessing the webserver (including the wiki) through the web. To access all other resources, you need to be connected in one of three ways:

1. By a wired ethernet connection on the Rice campus, including through wired workstations at the McNair Center
2. Through the Rice Owls wireless network (note: the Rice Visitor network will not allow you access)
3. From off-campus (or through Rice Vistor, etc.) but through the VPN. Connection details are available here: https://docs.rice.edu/confluence/display/ITDIY/Connecting+to+the+Rice+network+with+VPN

Connecting to the RDP Server

To connect to the RDP Server, you will need a user account on the RDP server. If you don't have one, see Ed.

For PCs

1. Go to the Start menu and search for and then open the "Remote Desktop Connection" application
2. In the new window, enter the RDP machine's IP address (128.42.44.182) into the text box labeled "Computer"
3. Click "Show Options" and then enter ad.mcnaircenter.org\user (where user should be replaced with your username on the RDP server) as the username
4. Click Connect and type in your password when prompted
5. You should now be able to access the RDP machine from the resulting window

For Macs

1. To connect from Mac to Windows, download the Microsoft Remote Desktop App off the Mac App store.
2. Open the app and go to Create / Edit Remote Desktops.
3. On the Connection Center screen click New. A new window should pop up prompting you for several fields of information. Here are the important ones:

• PC name: this should be the RDP machine's IP address (128.42.44.182)
• Credentials: The user name and password to be used for the RDP server. Your username should be ad.mcnaircenter.org\user (where user is replaced with your username on the RDP server) and your password is your password on the RDP server.

1. Save the remote desktop and you should be able to access it from the Microsoft Remote Desktop App.

Mapping the Network Drive

To map to a network drive, you will need a user account on the RDP server. If you don't have one, see Ed.

For PCs

1. Open File Explorer on your PC and click "This PC"
2. Look for the "Computer" tab in the menu at the top of the window and choose "Map Network Drive"
3. In the new window, choose any empty drive letter (for example, Z:) and for the folder, type \\128.42.44.182\bulk
4. Check the "Connect using different credentials" checkbox
5. Click Finish and enter your username (in the form ad.mcnaircenter.org\user where user is replaced with your username on the RDP server) and password when prompted
6. You should be able to see the network drive on your computer now.

For Macs

1. Go into the OS X Finder and hit Command+K which should bring up the "Connect to Server" menu.
2. In that menu, enter smb://128.42.44.182/bulk into the server address and click Connect.
3. Enter your username (in the form ad.mcnaircenter.org\user where user is replaced with your username on the RDP server) and password on the RDP server.
4. The drive should now appear on your Finder menu sidebar.

Connecting to the Database Server

You can map the database server's drive in the same way that you map the bulk drive on the RDP server. The details are:

IP address: 128.42.44.181
username (you don't need to specify the domain): researcher 
password: 9million

You can connect the database server from your laptop or from the RDP. For connecting from the RDP, follow the connection for connecting from windows.

Connecting to the dbase server from windows

• Download putty.exe (its free)
• Connect to 128.42.44.181 (save your connection for later)
• Username: researcher
• Password: 9million

Connecting to the dbase server from mac

• Open a terminal
• Connect using (on the first time you have to accept the certificate):

ssh 128.42.44.181

• Same username and password as above

Using the Dbase server

Change directory to your project's directory and the open a connection to the dbase server:

cd /bulk/whatever
psql yourdb

To create a dbase, just type:

createdb yourbdname

Using the Normalize Fixed Width Script

There is a very useful script that will take text files with lots of spaces and turn them into tab delimited text files, this file can currently be found at "E:\Processing\SDC Data." You can access it by

1. RDP onto the server and open Windows Powershell. It is on the taskbar at the bottom of the screen with a blue square icon with an arrow in it.
2. type in "cd 'E:\Processing\SDC Data' "

OR

1. Open File Explorer
2. Look in bulk (E:) then Processing then SDC Data

There you should be able to find the script NormalizeFixedWIdth.pl. In order to run the script you must RDP onto the server and use PowerShell. There are 2 easy ways to get your file into the same folder as this script

1. You could copy the script into the folder with the files you want to normalize
2. You could copy the files you want to normalize into "E:\Processing\SDC Data" and then take it out once you are done normalizing it

Once you have performed one of the above steps you can use the script to normalize your text file. To do this:

1. Open windows PowerShell
2. use the cd (change directory) command to navigate to the place you put both the text file and the script, if you moved your text file, use "cd 'E:\Processing\SDC Data' ". The proper way to use this command is to type "cd (name of folder you want to move into without parentheses around it)" into PowerShell.
3. PowerShell should say "PS E:\Processing\SDC Data>" or a different filepath if you moved the script.
4. Now in order to use the script type "perl NormalizeFixedWidth.pl -file='name of file to normalize (make sure to include the single quotes on the outside and the file extension or the script won't work)' "

Server Backup Policy

Backups are written to an external usb HDD at midnight everyday. Backup logs are also updated.

1. For linux/unix machines this is done by invoking backup scripts from cron (see /etc/cron.tab for execution set-up)
2. For windows machines we use the built in Windows Server Backup tool.

RDP server

Current backup policy is to backup complete \bulk folder.

To change/schedule a new policy use the Windows Server Backup tool.

1. 6/15/2016
   1. this corresponds to full E:\ volume.

WEB server

MySQL databases

Current backup policy is to backup all MySQL databases on the server.

Backup script: /root/scripts/web_mysqldump_backup.plx
Backup destination path: /media/usb_backup/mysqldump/
Backup log: /var/log/web_mysqldump_backup.log.gz

The script uses mysqldump and rsync for backup.

1. 6-13-2016
   1. backup destination paths are hard-coded in the script.
   2. TO DO: add command line options like --backup-destination= remote/local, --verbose/quiet
   3. redo this script like for database server (see below) because its cleaner implementation.

To see more information see perldoc

perldoc /root/scripts/web_mysqldump_backup.plx

/var/www

This contains mediawiki, blogs and other stuff running on the apache webserver.

Backup script: /root/scripts/www_rsync_backup.plx
List of files to back up: /root/scripts/www_rsync_backup.list
Backup destination path: /media/usb_backup/www
Backup log: /var/log/www_rsync_backup.log

The backup path reflects directory structure on host(source) starting at /.

1. 6-16-2016
   1. The script rsyncs the files from host to backup.
   2. We should probably save snapshots weekly/monthly..

/home

This contains home directories for all users.

Backup script: /root/scripts/home_rsync_backup.plx
List of files to back up: /root/scripts/home_rsync_backup.list
Backup destination path: /media/usb_backup/home
Backup log: /var/log/home_rsync_backup.log

The backup path reflects directory structure on host(source) starting at /.

1. 6-16-2016
   1. The script rsyncs the config files from host to backup.
   2. We should probably save snapshots weekly/monthly..

Database server

PostgreSQL databases

Current backup policy is to backup most PostgreSQL databases on the server. We ignore a few test databases.

Backup script: /root/scripts/db_pgdump_backup.plx
List of databases to back up: /root/scripts/db_pgdump_backup.list
Backup destination path: /media/backup_usb/pgdump/
Backup log: /var/log/db_pgdump_backup.log

The script uses pg_dump for backup. All databases are then collected in a tar archive.

1. 6-15-2016
   1. backup destination paths are hard-coded in the script.
   2. TO DO: add command line options like --backup-destination= remote/local, --verbose/quiet

To see more information see perldoc

perldoc /root/scripts/db_pgdump_backup.plx

6-16-2016: Above method is a very simple backup. We should do something like using barman which provides enterprise backup control.

PostgreSQL config backup

Backup these files

/var/postgresql/data/postgresql.conf : This file is read by psql on server startup.
/var/postgresql/data/postgresql.auto.conf : This is also read at server startup. Should not be edited manually. Config options are set with ALTER SYSTEM command.
/var/postgresql/data/pg_hba.conf:  Contains psql user account information.
/var/postgresql/data/pg_ident.conf : Contains mapping from psql user accounts to system accounts.
We probably don't need last two files. Better to recreate them when restoring the database.

Backup script: /root/scripts/psql_cfg_rsync_backup.plx
List of config files to back up: /root/scripts/psql_cfg_rsync_backup.list
Backup destination path: /media/backup_usb/postgresql_config
Backup log: /var/log/psql_cfg_rsync_backup.log

The backup path reflects directory structure on host(source) starting at /.

1. 6-16-2016
   1. We rsync the config files from host to backup.
   2. We should probably save snapshots weekly/monthly..