7,612
edits
Changes
Jump to navigation
Jump to search
*[[Server Backup Policy]]
*[[Power Backup]]
*[[Progress tracking platform]]
*[[Server maintenance FAQ]]
*[[NLP Infrastructure]]
*[[RDP Compute Performance]]
no edit summary
Server Build notes:
*[[Computer Build]] -- Contains hardware spec
*[[RDP Documentation]] (IP Address: 128.42.44.182)
*[[Web Server Documentation]] (IP Address: 128.42.44.180)
*[[Database Server Documentation]] (IP Address: 128.42.44.181)
*[[Test Web Server Documentation]] (IP Address: 128.42.44.22)
*[[Power Backup]]
Other useful resources:
*[[Haas PhD Server Configuration]]
*[[Software Repository]]
*[[Patent Data Processing - SQL Steps]]
Wishlist:
*[[McNair Security Policy]]
[[Semantic Form Data structure]]
# The drive should now appear on your Finder menu sidebar.
===Connecting to the Database Server===
You can map the database server's drive in the same way that you map the bulk drive on the RDP server. The details are:
createdb yourbdname
=== Using the Normalize Fixed Width Script ===
# PowerShell should say "PS E:\Processing\SDC Data>" or a different filepath if you moved the script.
# Now in order to use the script type "perl NormalizeFixedWidth.pl -file='name of file to normalize (make sure to include the single quotes on the outside and the file extension or the script won't work)' "
== Server Backup Policy ==
Backups are written to an external usb HDD at midnight everyday. Backup logs are also updated.
# For linux/unix machines this is done by invoking backup scripts from cron (see /etc/cron.tab for execution set-up)
# For windows machines we use the built in Windows Server Backup tool.
=== RDP server ===
Current backup policy is to backup complete \bulk folder.
To change/schedule a new policy use the Windows Server Backup tool.
#6/15/2016
##this corresponds to full E:\ volume.
=== WEB server ===
==== MySQL databases ====
Current backup policy is to backup all MySQL databases on the server.
Backup script: /root/scripts/web_mysqldump_backup.plx
Backup destination path: /media/usb_backup/mysqldump/
Backup log: /var/log/web_mysqldump_backup.log.gz
The script uses mysqldump and rsync for backup.
#6-13-2016
##backup destination paths are hard-coded in the script.
##TO DO: add command line options like --backup-destination= remote/local, --verbose/quiet
##redo this script like for database server (see below) because its cleaner implementation.
To see more information see perldoc
perldoc /root/scripts/web_mysqldump_backup.plx
==== /var/www ====
This contains mediawiki, blogs and other stuff running on the apache webserver.
Backup script: /root/scripts/www_rsync_backup.plx
List of files to back up: /root/scripts/www_rsync_backup.list
Backup destination path: /media/usb_backup/www
Backup log: /var/log/www_rsync_backup.log
The backup path reflects directory structure on host(source) starting at /.
#6-16-2016
## The script rsyncs the files from host to backup.
## We should probably save snapshots weekly/monthly..
==== /home ====
This contains home directories for all users.
Backup script: /root/scripts/home_rsync_backup.plx
List of files to back up: /root/scripts/home_rsync_backup.list
Backup destination path: /media/usb_backup/home
Backup log: /var/log/home_rsync_backup.log
The backup path reflects directory structure on host(source) starting at /.
#6-16-2016
## The script rsyncs the config files from host to backup.
## We should probably save snapshots weekly/monthly..
=== Database server ===
==== PostgreSQL databases ====
Current backup policy is to backup most PostgreSQL databases on the server. We ignore a few test databases.
Backup script: /root/scripts/db_pgdump_backup.plx
List of databases to back up: /root/scripts/db_pgdump_backup.list
Backup destination path: /media/backup_usb/pgdump/
Backup log: /var/log/db_pgdump_backup.log
The script uses pg_dump for backup. All databases are then collected in a tar archive.
#6-15-2016
##backup destination paths are hard-coded in the script.
##TO DO: add command line options like --backup-destination= remote/local, --verbose/quiet
To see more information see perldoc
perldoc /root/scripts/db_pgdump_backup.plx
6-16-2016: Above method is a very simple backup. We should do something like using
[https://www.digitalocean.com/community/tutorials/how-to-back-up-restore-and-migrate-postgresql-databases-with-barman-on-centos-7 barman] which provides enterprise backup control.
==== PostgreSQL config backup ====
Backup these files
/var/postgresql/data/postgresql.conf : This file is read by psql on server startup.
/var/postgresql/data/postgresql.auto.conf : This is also read at server startup. Should not be edited manually. Config options are set with ALTER SYSTEM command.
/var/postgresql/data/pg_hba.conf: Contains psql user account information.
/var/postgresql/data/pg_ident.conf : Contains mapping from psql user accounts to system accounts.
We probably don't need last two files. Better to recreate them when restoring the database.
Backup script: /root/scripts/psql_cfg_rsync_backup.plx
List of config files to back up: /root/scripts/psql_cfg_rsync_backup.list
Backup destination path: /media/backup_usb/postgresql_config
Backup log: /var/log/psql_cfg_rsync_backup.log
The backup path reflects directory structure on host(source) starting at /.
#6-16-2016
## We rsync the config files from host to backup.
## We should probably save snapshots weekly/monthly..
[[category:McNair Admin]]
[[admin_classification::IT Build| ]]
