7,612
edits
Changes
Jump to navigation
Jump to search
=====PostGIS Issues=====
So, option 2 is to shutdown version 10, move version 12 to ======Switching over the right port, and load up the dbases... Then wipe out the old datastructure and move the new one to the right location. installations======
For To fix the configuration of the servers built for the McNair Center, see the old [[Center IT]] page or the pages belowmain alias in apache:*[[Database Server Documentation]] vi /etc/apache2/sites-available/edegan.com.conf*[[RDP Documentation]]**[[Server Backup Policy]]**[[Power Backup]] Change:*[[Test Web Server Documentation]] Alias /blog /var/www/html/blog/ RewriteRule ^/*$ %{DOCUMENT_ROOT}/blog/index.php [[Web Server Documentation]L]**[[Wiki Configuration]]**[[Security on the wiki]] systemctl reload apache2
Some of I also needed to:#Perform a backup (I really should have done this information is still useful!before moving the site but hey...)#Fix the images etc. (they initially fine... but I came to suspect that they were cached and an HTML inspected revealed the wrong URLs)
→Private Changes
We then had to fix the passwords in the dbase:
#Note: change passwords from hints hint before running
mysql
SELECT User, Host, Password FROM mysql.user;
UPDATE mysql.user SET Password = PASSWORD('tsnhint') WHERE User = 'root'; UPDATE mysql.user SET Password = PASSWORD('tsnhint') WHERE User = 'debian-sys-maint'; UPDATE mysql.user SET Password = PASSWORD('tsnhint') WHERE User = 'mcnair_wp';
FLUSH PRIVILEGES;
To do the front page, I copied the source of [https://www.mediawiki.org/w/index.php?title=Template:Main_page&action=edit Template:Main page] from mediawiki to a page (called Test) and created [Template:Main page/styles.css] using [https://www.mediawiki.org/w/index.php?title=Template:Main_page/styles.css&action=edit mediawiki's code]. Then I rejigged the contents of the page!
The only minor but non-obvious change, was that I used h2 headings inside each mainpage_box, rather than h3's. As a consequence, I needed to add the following to [[Template:Main_page/styles.css]]:
.mainpage_box h2 {
border-bottom: none;
}
.mainpage_box h2 .header_icon {
margin-right: 5px;
}
Old instructions[https://www.mediawiki.org/wiki/Topic:R1j08xhjgrtkpj6q] suggest using [[Special:ExpandTemplate]] on mediawiki's wiki, with the input text <nowiki>{{:MediaWiki}}</nowiki>. But this isn't necessary as the template doesn't need expanding in its current (at the time of writing) incarnation. Naturally, the page works well on MobileFrontend.
And the wiki now seems happy!
I also checked postgres and everything seemed ok:
su researcher
pg_dump -Fc stockmarket > stockmarket_Fc_20201023.dump
pg_dump -Fc vcdb4 > vcdb4_Fc_20201023.dump
#FAILED!
The second backoff failed:
') AS attfdwoptions FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t ON a.atttypid = t.oid WHERE a.attrelid = '19998614'::pg_catalog.oid AND
a.attnum > 0::pg_catalog.int2 ORDER BY a.attnum
======Postgres Upgrade Attempt (Failed)======
My changes weren't substantial, so I proceeded with an upgrade. First I checked to see if I had postgres12 installed and listening on another port or not:
#This failed too - there are unmet dependencies and they are 'not installable'.
So, I took the alternative approach of changing the data folders [https://www.tutorialspoint.com/how-to-change-postgresql-data-folder-location-on-ubuntu-16-04].
The plan:
#Take version 10 offline
#Move version 10's data to a new location (/var/lib/postgresql/10/main)
#Switch the ports of versions 10 and 12
#Move version 12's data to /data
#Put version 12 online
#Load up the data in version 12!
#Optionally wipe out the old installation
Shut it down:
pg_ctlcluster 12 master start
pg_lsclusters
#The cluster
Ver Cluster Port Status Owner Data directory Log file
10 main 5432 online postgres /data/postgres /var/log/postgresql/postgresql-10-main.log
12 master 5433 down online postgres /var/lib/postgresql/12/master /var/log/postgresql/postgresql-12-master.logg systemctl stop postgresql systemctl status postgresql Edit the config files: vi /etc/postgresql/10/main/postgresql.conf data_directory = '/var/lib/postgresql/10/main' port = 5433 vi /etc/postgresql/12/master/postgresql.conf data_directory = '/data/postgres' port = 5432 listen_addresses = '*' #While we are here do some performance tuning: shared_buffers = 512MB huge_pages = try temp_buffers = 8G work_mem = 4GB maintenance_work_mem = 64 effective_cache_size = 384GB #Note that I didn't reduce the number of connections (and the max_wal_senders, which must be < max connections), or change max_stack_depth (which gives an error if you set it too high) vi /etc/postgresql/12/master/pg_hba.conf Copy over the config to allow access from inside the network Move the data: df #to check diskspace rm -R /var/lib/postgresql/10/main #Note that none of the config files in here were valid (though you should check this is true before you do it!) rsync -av /data/postgres / /var/lib/postgresql/10/main #Takes awhile, but make sure it is all done before the next step rm -R /data/postgres rsync -av /var/lib/postgresql/12/master / /data/postgres systemctl start postgresql pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log 12 master 5432 online postgres /data/postgres /var/log/postgresql/postgresql-12-master.log Do the installs for some extensions: apt-get install postgresql-12-plr apt-get install postgresql-plperl-12 postgresql-plpython3-12 Check it all works: psql postgres CREATE EXTENSION postgis; CREATE EXTENSION plr; CREATE EXTENSION plperl; CREATE EXTENSION plpython3u; \dx List of installed extensions Name | Version | Schema | Description---------+---------+------------+--------------------------------------------------------------------- plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 3.0.0 | public | PostGIS geometry, geography, and raster spatial types and functions (2 rows) Make the user: createuser --interactive researcher Then restore the databases (as researcher in /bulk/backup): createdb stockmarket pg_restore -Fc -d stockmarket stockmarket_Fc_20201023.dump createdb vcdb4 pg_restore -Fc -d vcdb4 vcdb4_Fc.dump The restore threw some errors related to not having extension plpythonu, but otherwise seemed fine. The issue seems to be that pythonu is python2[https://www.postgresql.org/docs/12/plpython-python23.html], and python2 is not available for postgres 12 (it might be here: https://wiki.postgresql.org/wiki/Apt): apt-cache search ".*plpython*.*"
====Other Fixes====
Once I'm all done with the theme etc., I can uncomment the following from wp-config.php
define('DISALLOW_FILE_EDIT', true);
{{Colored box|title=Notice|content=With hardened permissions, you won't be able to update Wordpress from the dashboard. To fix this, set ownership of the entire wordpress directory to www-data:www-data (i.e., chown -R www-data:www-data blog/), run the update, and then revert the ownership to root (or some other account).}}
====Redesign====
define( 'WPMS_SMTP_PASS', 'password generated by Google' );
define( 'WPMS_SMTP_AUTOTLS', true );
=====Author Comments=====
The blog supports multiple authors and by default, Wordpress emails an author whenever one of their posts gets a comment. If you'd like to disable author comment emails but keep the moderator emails, there's a simple fix:
'''Just go to wp-admin/options.php and set 'comments_notify' to 0.''' (See https://codex.wordpress.org/Option_Reference)
More complicated methods involve writing your own plugin [https://wordpress.stackexchange.com/questions/150125/disabling-comment-notifications-for-post-author] to refine wp_new_comment_notify_postauthor[https://developer.wordpress.org/reference/functions/wp_new_comment_notify_postauthor/] or changing the hooks[https://developer.wordpress.org/reference/hooks/notify_post_author/] used in wp-includes/comment.php:
$maybe_notify = apply_filters( 'notify_post_author', $maybe_notify, $comment_ID );
====Social Media Integration====
This is mentioned in a comment on a topic page, though presumably for an earlier version: https://www.mediawiki.org/wiki/Topic:Syxow0why4c0cvvm
====Another Issue==== Interestingly, I started getting a message from Google Chrome whenever I went to post wiki entries saying: "The information you’re about to submit is not secure". There's an option to "Proceed anyway" or "Go back". This started after I had MultiTail running viewing apache's logs, but I couldn't see, beyond some kind of file lock examination, how it could work. I figured that it was a coincidence and something else might have happened. My first thought was that my SSL certificate might have expired. However, the certificate looks valid and good, and the issue survived a reboot. By inspecting the webpages (in Chrome) and then reviewing the Console, I could see that it was caused by a mixed content problem: Mixed Content: The page at '<URL>' was loaded over HTTPS, but requested an insecure font '<URL>'. This request has been blocked; the content must be served over HTTPS. It seemed that I somehow have some font addresses hardcoded somewhere: Mixed Content: The page at ... was loaded over HTTPS, but requested an insecure font 'http://128.42.44.180/mediawiki/resources/assets/fonts/BonvenoCF-Light.otf'. This request has been blocked; the content must be served over HTTPS. The copy of Chrome on my desktop must somehow have been upgraded? Or something else changed to cause a change in behavior... The IP is from the old web server at the McNair Center, suggesting that when I migrated the McNair database into the new wiki, I migrated this issue. (Note that it doesn't appear to be something hardcoded into a .css file, or similar -- I can't find any trace on the filesystem and besides, this wiki was built from a fresh install.) I found the URLs hardcoded in [[MediaWiki:Common.css]] (it must have been moved with the last big batch of pages and I somehow didn't notice!) but then couldn't edit it! It seems that following [https://www.mediawiki.org/wiki/MediaWiki_1.32/interface-admin Mediawiki 1.32], the rights to edit the interface were separated out, and users now need the editinterface right to change anything in the Mediawiki namespace. So, I went to [[Special:UserRights]] and gave myself permission. Then I edited the page, which changed the look-and-feel of my editor (I have no idea why), removed the consol messages, but left the problem (even after ctrl-shift-r cache flush on Chrome). ===Install VSFTPD===
With the security restrictions on wordpress, I now need an FTP server to get files for themes, plugins, etc. I like VSFTPD, as its simple, secure, and has a nice standalone config. Old documentation on an earlier install on the old [[Wordpress Blog Site (Tool)]] page. Instructions are here: https://linuxconfig.org/how-to-setup-ftp-server-on-ubuntu-20-04-focal-fossa-linux
userdel blog
===Final Configuration Changesto Apache===
Lock down apache somewhat further (as now there are directories that shouldn't be listable, etc.)
Note: Don't change the DocumentRoot to the blog, as this will destroy the design of the wiki. The last rewrite rule will decide the default site!
==Old machines=Changing the Wordpress url=== It seems likely that some Reddit bots are automatically blocking postings with the word blog in their URL. So I decided to move my Wordpress installation from /blog to /journal. Note that I briefly tried 'article' (singular to save letters). I also considered 'paper', which is shorter but not quite right. Likewise study, etc. Also digest (which was a close second), review, bulletin, and pub (which was just too ambiguous). The problem with article is that although it looks good for article URLS, the landing URL is then www.edegan.com/article. A collection word is better. And journal appeals to the academic in me. To do this takes about 10 minutes (see https://wordpress.org/support/article/moving-wordpress/). You have to:#Make the change in Wordpress through wp-admin (you have to do this first!) - Set both the Wordpress Address (URL) and Site Address (URL) fields#Move the directory to the new name (at this point I could access www.edegan.com/article)#Fix aliases in apache and the default subdomain for landing (see below)#Fix the permalinks (update the .htaccess file)#Fix the links hardcoded in menus#(Create and) load up a new icon image (it is set under Appearance -> Customize -> Site Identity)#Run any updates etc.#Reconnect Google site kit#Rerun Yoast SEO optimizer
Do the backup (with a user that has process privileges): cd /bulk/backups mysqldump --add-drop-table -h localhost -u user -p wordpress | gzip > wordpress-20210209.sql.gz or mysqldump --add-drop-table -h localhost -u user -p wordpress > wordpress-20210209.sql Then install and use Velvet Blues (see https://www.wpbeginner.com/plugins/how-to-update-urls-when-moving-your-wordpress-site/) and use it to change the URLs (use https:// as a page inspection shows that this is correct) ===Another Change to the Wordpress URL and some updates === As root get the mysql dbase details: mysql -p #hint: tsn select host, user from mysql.user; show databases; Read mail: less /var/mail/$(whoami) Look in wp-config.php for dbase, user, pword etc. cat /var/www/html/journal/wp-config.php Back up the dbase mysqldump --add-drop-table -h localhost -u root -p wordpress > wordpress-20220814.sql Change settings (on https://www.edegan.com/journal/wp-admin/options-general.php)* WordPress Address (URL)** https://www.edegan.com/articles* Site Address (URL)** https://www.edegan.com/articles Move the folder: cd /var/www/html/ mv journal articles Reload: https://www.edegan.com/articles/wp-admin/options-general.php? Fix the .htaccess file to do the permalinks cd articles vi .htaccess #change 2 instances of journal to articles Fix the links in the menu on https://www.edegan.com/articles/wp-admin/customize.php?* Change links in menus! (2 instances - then publish) Upgrade php to 7.4 sudo apt-add-repository ppa:ondrej/php apt update apt install -y php7.4 php7.4-cli php7.4-common php7.4-fpm apt install -y php7.4-mysql php7.4-dom php7.4-simplexml php7.4-ssh2 php7.4-xml php7.4-xmlreader php7.4-curl php7.4-exif php7.4-ftp php7.4-gd php7.4-iconv php7.4-imagick php7.4-json php7.4-mbstring php7.4-posix php7.4-sockets php7.4-tokenizer apt install -y php7.4-mysqli php7.4-pdo php7.4-sqlite3 php7.4-ctype php7.4-fileinfo php7.4-zip php7.4-exif a2dismod php7.3 a2enmod php7.4 a2enmod proxy_fcgi setenvif a2enconf php7.4-fpm systemctl reload apache2 Update wordpress* Note that I have wordpress chown -R root:root for articles and then www-data:www-data for wp-content.* Didn't read: https://www.edegan.com/articles/wp-admin/update-core.php?action=do-core-upgrade Do the update /var/www/html# chown -R www-data:www-data articles/ run update in wp-admin * Update all the plugins* Update themes* Wordfence** Update .htaccess for extended protection. Reharden: chown -R root:root articles/ chown -R www-data:www-data articles/wp-content/ Fix the alias in apache! vi /etc/apache2/sites-available/edegan.com.conf Alias /journal /var/www/html/articles Alias /articles /var/www/html/article Alias /blog /var/www/html/blog RewriteRule ^/*$ %{DOCUMENT_ROOT}/articles/index.php [L] #Redirect the journal root to articles Redirect permanent /journal https://www.edegan.com/articles systemctl reload apache2 Plugins etc.* Site Kit by Google** Setup! (sign in using dredegan@gmail.com)* Yoast SEO** Rerun optimization Fix image links, etc.* I repointed the /journal and /blog aliases* Run the velvet blues plugin Fix the icon:* https://www.edegan.com/articles/wp-content/uploads/2021/02/edegandotcomslashjournal-LightGreyOnDarkBlue.png* In Z:\projects\WebDesign\LogosV2.xcf* Export as edegandotcomslasharticles-LightGreyOnDarkBlue.png* Set under Site Identity ===Useful tools=== ====Multitail==== I installed [https://www.vanheusden.com/multitail/manual.php Multitail]: apt-get multitail The [https://www.vanheusden.com/multitail/manual.php manual] is pretty weak, but the [https://www.vanheusden.com/multitail/examples.php examples] are good and the [https://www.vanheusden.com/multitail/features.php feature list] is excellent. Here's some useful commands to review log files: multitail -cS apache -ev "Bot" /var/log/apache2/access.log -ci white -e "Bot" -I /var/log/apache2/access.log multitail -cS apache -ev "Bot" -ev "bot" -ev "internal dummy connection" /var/log/apache2/access.log ====Traceroute==== apt install traceroute Note: [https://zmap.io/ Zmap] seems popular nowadays, based on traffic logs. ====Other==== Connect with smb from Mother to Father: smbclient //192.168.2.200/sharename -U Domainname/username Note that you need to specify the domain. Mount a thru connection: mount -t cifs -o user=username //192.168.2.200/sharename /mnt/father ln -s /mnt/father/whatever/ /bulk/whatever ==Old machines== For the configuration of the servers built for the McNair Center, see the old [[Center IT]] page or the pages below:*[[Database Server Documentation]]*[[RDP Documentation]]**[[Server Backup Policy]]**[[Power Backup]]*[[Test Web Server Documentation]]*[[Web Server Documentation]]**[[Wiki Configuration]]**[[Security on the wiki]] Some of this information is still useful! In addition, at UC Berkeley, Ed designed and built three machines - two postgresql database servers and a wiki server. The documentation is here:
*[[Haas PhD Server Configuration]]
*[[Posgres Server Configuration]] -- documents the build of postgres2
== [[Private Configuration]] ==
Some [[Private Configuration]] changes to the research computing setup are not recorded on the public wiki pages.
