Changes

44,412 bytes added , 5 February

This page describes the configuration of the ~~two~~ new research computing machines: '''Father''' (Windows Server 2019) and '''Mother''' (Ubuntu Server 20. ~~This configuration runs on our [[Research Computing Hardware]]~~04). Note that ~~the configuration of~~ the [[~~DIGITS DevBox~~RDP Software Configuration]] is describes the software installed on ~~its own page~~Father.

The hardware description and complete build notes and configuration information for '''Bastard''', our blisteringly fast, multi-GPU, A.I. estimation platform, are on the [[DIGITS DevBox]] page. The hardware descriptions for Father and Mother are on the [[Research Computing Hardware]] page. See also: [[Recovering Astarte]] for notes on moving content from an old mediawiki installation to a new one. ==Both machines(Father and Mother)==

===Fan Control===

sudo update-secureboot-policy --enroll-key

==RDP Server(Father)==

The RDP server runs Windows 2019 Server. It installs directly off the media on to the NVMe drive. Don't worry about the RAID array during the installation, we do that later.

*NVIDIA USBC Driver 1.1.27.831

==Dbase Server(Mother)==

The dbase server runs Ubuntu 18.04. You can mostly follow the instructions at https://www.pugetsystems.com/labs/hpc/The-Best-Way-To-Install-Ubuntu-18-04-with-NVIDIA-Drivers-and-any-Desktop-Flavor-1178/

Finally restart samba:

systemctl restart smbd

~~systemctl restart nmbd~~

Check it works:

We then had to fix the passwords in the dbase:

#Note: change passwords from ~~hints~~ hint before running

mysql

SELECT User, Host, Password FROM mysql.user;

UPDATE mysql.user SET Password = PASSWORD('~~tsn~~hint') WHERE User = 'root'; UPDATE mysql.user SET Password = PASSWORD('~~tsn~~hint') WHERE User = 'debian-sys-maint'; UPDATE mysql.user SET Password = PASSWORD('~~tsn~~hint') WHERE User = 'mcnair_wp';

FLUSH PRIVILEGES;

*Fix [[<haloacl-inclusion-denied>|Terms of Service]] on sign up page...

===Mediawiki Redux=== ====Mobile Front End====

It seems that the [https://www.mediawiki.org/wiki/Extension:MobileFrontend Extension:MobileFrontend] isn't working properly.

====Does php have mbstring support?====

First check the apache version:

So everything seems fine (the extension it is listed as loaded in Special:Version), but the interface still has obvious issues.

====Check the skin====

From Special:Version

And it looks like my version of mediawiki is too old to support [https://www.mediawiki.org/wiki/Skin:Minerva_Neue Minerva Neue]

===Upgrade mediawiki===

Essentially, follow instructions in [https://www.mediawiki.org/wiki/Manual:Upgrading Manual:Upgrading]:

====Backup====

I just moved everything to a different directory, backed off the dbase, and started again.

mysqldump --user=root --password=password > dbase.sql

====New Install====

Get a new version, put it in /var/lib/mediawiki (leaving the old shortcuts pointing there), then copy in the files.

php update.php

====Extensions==== =====Retrieve and ~~configure~~ include extensions=====

Get the bulk of them...

wget https://extdist.wmflabs.org/dist/extensions/GeoData-REL1_34-8a52fa4.tar.gz

tar -xzf GeoData-REL1_34-8a52fa4.tar.gz -C /var/lib/mediawiki/extensions

wget https://extdist.wmflabs.org/dist/extensions/NumberFormat-REL1_34-cf8a23e.tar.gz

tar -xzf NumberFormat-REL1_34-cf8a23e.tar.gz -C /var/lib/mediawiki/extensions

Do the semantic mediawiki install and config. Last time, we installed Semantic Mediawiki using composer, which is the preferred method. See [[Web_Server_Documentation]].

service apache2 restart

~~Also, add Special:MultiUpload|Upload multiple files to http://www.edegan.com/wiki/MediaWiki:Sidebar~~

====MathML====

I tried various methods to get MathML to work and always failed. It looks like the community bet on Mathoid working out, but there's been no development on it for 5 months now, and it looks dead. The good news~~, if~~ is that MathJax works just fine right out of the box:

git clone https://github.com/jmnote/SimpleMathJax.git

wfLoadExtension( 'SimpleMathJax' );

#$wgSmjInlineMath = [ [ "$", "$" ], [ "\$", "\$" ] ]; Note: the last line lets you demark math with LaTeX-like syntax. I disabled it, as I use $ signs way too much in other contexts.

====PDFEmbed====

or equivalently (but silently):

<nowiki>{{#set: Visible to=whitelist|Visible to group::=team}}</nowiki>

=====BibTeX=====

The BibTeX extension doesn't work anymore. Though it never really worked, so it's not much of a loss. I should probably build a replacement but I don't have the time right now.

=====Upload Multiple Files===== Although the Upload multiple files extension installed fine, it is unmaintained and seems to have an issue. I removed its line from LocalSettings.php and deleted its extension directory. I then installed [https://www.mediawiki.org/wiki/Extension:SimpleBatchUpload Simple Batch Upload] using a tarball: in mediawiki/installs wget https://github.com/ProfessionalWiki/SimpleBatchUpload/archive/1.6.0.tar.gz tar -xzf 1.6.0.tar.gz -C /var/lib/mediawiki/extensions mv /var/lib/mediawiki/extensions/SimpleBatchUpload-1.6.0 /var/lib/mediawiki/extensions/SimpleBatchUpload In LocalSettings.php: wfLoadExtension( 'SimpleBatchUpload' ); $wgSimpleBatchUploadMaxFilesPerBatch = ['*' => 10,]; I had previously added Special:MultiUpload|Upload multiple files to http://www.edegan.com/wiki/MediaWiki:Sidebar. I replaced it with a link to [[Special:BatchUpload]]. =====Allow SVG images===== See https://www.mediawiki.org/wiki/Manual:Image_administration#SVG. Essentially, add svg to $wgFileExtensions, then install and designate an image converter. I went with rsvg: apt-get install librsvg2-bin vi LocalSettings.php $wgSVGConverter = 'rsvg'; =====Add HitCounters===== wget https://extdist.wmflabs.org/dist/extensions/HitCounters-REL1_34-48dd6cb.tar.gz tar -xzf HitCounters-REL1_34-48dd6cb.tar.gz -C /var/lib/mediawiki/extensions vi ../LocalSettings.php wfLoadExtension( 'HitCounters' ); cd ../maintenance php update.php ====Change the Dbase==== I tried to 'rename' the dbase, creating a dedicated dbase user that has access to just the wiki's dbase, and resetting its password. in /bulk/backups: mysqldump --password olddbase > mediawiki.sql mysql -u olduser -p CREATE DATABASE wiki; mysql -u olduser -p wiki < mediawiki.sql mysql -u olduser -p CREATE USER 'wiki'@'localhost' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON wiki.* TO 'wiki'@'localhost'; FLUSH PRIVILEGES; This led to two different errors. First, the cloned database didn't seem to support Semantic Mediawiki somehow. And second, the new user didn't seem to work. This was true even if I gave them rights on the olddbase. So I abandoned the attempt. Clean up the attempt: mysql -u olduser -p DROP DATABASE wiki; DROP USER 'wiki'@'localhost'; I did put an .htaccess file in /var/lib/mediawiki to restrict access to LocalSettings.php, though I expect that this is redundant. <files LocalSettings.php> order allow,deny deny from all </files> ====Setting up for advanced template(s) import==== =====Robelbox===== Importing the Robelbox, or other, templates on mediawiki is tricky [https://stackoverflow.com/questions/678626/where-can-i-get-templates-for-mediawiki], at least the first time that you do it. Once you've got everything up and running to support templates (see above):#Find the template on another mediawiki installation#Go to Special:Export on that installation and export the template (but not its revision history)#Import the template on the wiki using Special:Import, set the interwiki prefix to something that designates the source, like en for English Wikipedias. I did this for the [https://ts.wikipedia.org/wiki/Template:Robelbox Robelbox template], which I got from https://en.wikiversity.org/wiki/Special:Export, however, it wasn't useable and I couldn't work out why. I ultimately deleted Robelbox, having found better boxes (see below) but I expect that my process for fixing the later issues would have sorted out the problems here too. =====Fixing Template Issues===== I got Template:Box-round from mediawiki.org: https://www.mediawiki.org/wiki/Template:Box-round. It required installation of TemplateStyles, which in turn might need JsonConfig: In mediawiki/installs wget https://extdist.wmflabs.org/dist/extensions/TemplateStyles-REL1_34-c4d6f25.tar.gz tar -xzf TemplateStyles-REL1_34-c4d6f25.tar.gz -C /var/lib/mediawiki/extensions wget https://extdist.wmflabs.org/dist/extensions/JsonConfig-REL1_34-f877d87.tar.gz tar -xzf JsonConfig-REL1_34-f877d87.tar.gz -C /var/lib/mediawiki/extensions Add to LocalSettings.php wfLoadExtension( 'TemplateStyles' ); wfLoadExtension( 'JsonConfig' ); Now [[Template:Box-round]] works fine but, like [[Template:Tl]], has an error message on its page about JsonConfig being missing. (Note that Template:T1 previously said [[Module:TNT]] was missing, so I got it from www.mediawiki/w/Module:TNT). JsonConfig seems fine and shows in [[Special:Version]]. I added: $wgJsonConfigEnableLuaSupport = true;which gave: Lua error: bad argument #1 to "get" (not a valid title). So instead, I put all the following into LocalSettings.php to configure JsonConfig (see [https://www.mediawiki.org/wiki/Topic:Vdtgrltwf1e5bmdx]): // Safety: before extension.json, these values were initialized by JsonConfig.php if ( !isset( $wgJsonConfigModels ) ) { $wgJsonConfigModels = []; } if ( !isset( $wgJsonConfigs ) ) { $wgJsonConfigs = []; } $wgJsonConfigEnableLuaSupport = true; // https://www.mediawiki.org/wiki/Extension:JsonConfig#Configuration $wgJsonConfigModels['Tabular.JsonConfig'] = 'JsonConfig\JCTabularContent'; $wgJsonConfigs['Tabular.JsonConfig'] = [ 'namespace' => 486, 'nsName' => 'Data', // page name must end in ".tab", and contain at least one symbol 'pattern' => '/.\.tab$/', 'license' => 'CC0-1.0', 'isLocal' => false, ]; // Enable Tabular data namespace on Commons - T148745 $wgJsonConfigInterwikiPrefix = 'commons'; $wgJsonConfigs['Tabular.JsonConfig']['remote'] = [ 'url' => 'https://commons.wikimedia.org/w/api.php' ]; Then I copied over Module:Documentation/styles.css from mediawiki.org, commenting out the background image in line 168. And everything seems to work fine... I got the following templates from mediawiki (via [https://www.mediawiki.org/wiki/Special:Export mediawiki's Special:Export] to get dependencies):*Template:Colored box*Template:Navbox*Template:Help box*Template:Side box*Template:Note Note that this overwrote Template:Tl, Template:TNT, and others that I resolved issues with previously. Now a new set of issues has emerged. These include JsonConfig problems (again), and template loops (which I think are coming from Module:Template translation), and missing dependencies (e.g, Template:Mbox and Module:Color contrast). Moreover, most of the templates render the if and other conditional logic statements, rather then executing them. I installed ParserFunctions (which I should have done before) and it solved everything outstanding! In mediawiki/installs wget https://extdist.wmflabs.org/dist/extensions/ParserFunctions-REL1_34-4de6f30.tar.gz tar -xzf ParserFunctions-REL1_34-4de6f30.tar.gz -C /var/lib/mediawiki/extensions Add to LocalSettings.php wfLoadExtension( 'ParserFunctions' ); $wgPFEnableStringFunctions = true; ====The Front Page==== To do the front page, I copied the source of [https://www.mediawiki.org/w/index.php?title=Template:Main_page&action=edit Template:Main page] from mediawiki to a page (called Test) and created [Template:Main page/styles.css] using [https://www.mediawiki.org/w/index.php?title=Template:Main_page/styles.css&action=edit mediawiki's code]. Then I rejigged the contents of the page! The only minor but non-obvious change, was that I used h2 headings inside each mainpage_box, rather than h3's. As a consequence, I needed to add the following to [[Template:Main_page/styles.css]]: .mainpage_box h2 { border-bottom: none; } .mainpage_box h2 .header_icon { margin-right: 5px; } Old instructions[https://www.mediawiki.org/wiki/Topic:R1j08xhjgrtkpj6q] suggest using [[Special:ExpandTemplate]] on mediawiki's wiki, with the input text <nowiki>{{:MediaWiki}}</nowiki>. But this isn't necessary as the template doesn't need expanding in its current (at the time of writing) incarnation. Naturally, the page works well on MobileFrontend. ====Mass Edit==== The Mass edit page contains several examples, two of which show "Undefined Control Sequence" errors. These examples use backslashed square brackets (I can't even write them on the wiki using nowiki tags), which have issues because of Extension:SimpleMathjax. Regardless, the extension seems to work just fine! ===Update Linux===

Get the system up to date:

But I should really do a full backup and everything first, so that isn't going to happen today.

==~~Reviewing~~ =Wordpress===

====Overview====

Rather than trying to update wordpress, I think it best to install the latest version and use the old dbase. This will likely cause problems with images... but we made several suboptimal choices when we built the last version, including using a non-standard theme and customizing it in a way that prevented updates.

Outside of the defaults, [https://wordpress.org/themes/oceanwp/ OceanWP] is eCommerce oriented but looks good and is very popular. [https://wordpress.org/themes/neve/ Neve] sits between OceanWP and [https://wordpress.org/themes/generatepress/ GeneratePress], which has a more magazine/news focus, and all three take advantage of the new block editor ([https://wordpress.org/gutenberg/ Gutenberg]), which was introduced in WPv5 (initial release in 2018).

====Pre-install====

Check PHP and MySQL. I need PHP >=7.3 and MySQL >=5.6 but:

I might be able to upgrade my version of PhP without upgrading Ubuntu (see https://linuxize.com/post/how-to-install-php-on-ubuntu-18-04/). However, it is probably a good idea to just fix everything...

===Upgrading LinuxDistro===

So it turns out that I shouldn't have done that last update... I do have an Xwindows Server on the box, running Gnome, and now I can't log in using the GUI on the box itself (it loops back to the login screen). This box doesn't contain the GPUs, just the database server, so the GUI isn't key, but it would be nice to have it working again. Hopefully, an upgrade will fix that, as well as other issues.

====Backing off====

First, mount the USB drive. Find what's mounted and what the dev is:

umount /media/usb

====Do the upgrade====

Run:

I let it replace postgres10 but it still gave me an "Obsolete Major Version" warning on postgres (I said ok).

====Address the upgrade issues====

The first casualty of the upgrade was the networking configuration. You'd think that developers would have figured that one out, as remote upgrades would leave boxes DOA until someone could get physical access. Nevertheless, the fix is straight forward.

And the wiki now seems happy!

=====PostGIS Issues=====

I also checked postgres and everything seemed ok:

su researcher

All my extensions report back.

Update: It seems something did go wrong. Just because the extensions report back doesn't mean they work! When I try to run queries that use PostGIS, I get: SQL Error [58P01]: ERROR: could not access file "$libdir/postgis-2.4": No such file or directory I tried updating the extension (I'm pretty sure that I'm running 2.4.3): ALTER EXTENSION postgis UPDATE TO "2.4.3"; But that didn't fix anything. I checked the versions: select version(); PostgreSQL 10.14 (Ubuntu 10.14-0ubuntu0.18.04.1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit --So somehow I'm still running version 10! select PostGIS_full_version(); ERROR: could not access file "$libdir/postgis-2.4": No such file or directory CONTEXT: SQL statement "SELECT public.postgis_lib_version()" PL/pgSQL function postgis_full_version() line 25 at SQL statement So I backed off the data from the two dbases that I'd used since the upgrade: pg_dump -Fc stockmarket > stockmarket_Fc_20201023.dump pg_dump -Fc vcdb4 > vcdb4_Fc_20201023.dump #FAILED! The second backoff failed: pg_dump: [archiver (db)] query failed: ERROR: could not access file "$libdir/postgis-2.4": No such file or directory pg_dump: [archiver (db)] query was: SELECT a.attnum, a.attname, a.atttypmod, a.attstattarget, a.attstorage, t.typstorage, a.attnotnull, a.atthasdef, a.attisdropped, a.attlen, a.attalign, a.attislocal, pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, array_to_string(a.attoptions, ', ') AS attoptions, CASE WHEN a.attcollation <> t.typcollation THEN a.attcollation ELSE 0 END AS attcollation, a.attidentity, pg_catalog.array_to_string(ARRAY(SELECT pg_catalog.quote_ident(option_name) || ' ' || pg_catalog.quote_literal(option_value) FROM pg_catalog.pg_options_to_table(attfdwoptions) ORDER BY option_name), E', ') AS attfdwoptions FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t ON a.atttypid = t.oid WHERE a.attrelid = '19998614'::pg_catalog.oid AND a.attnum > 0::pg_catalog.int2 ORDER BY a.attnum ======Postgres Upgrade Attempt (Failed)====== My changes weren't substantial, so I proceeded with an upgrade. First I checked to see if I had postgres12 installed and listening on another port or not: locate postgres ls /usr/bin/postgres dpkg --get-selections | grep postgres pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 10 main 5432 online postgres /data/postgres /var/log/postgresql/postgresql-10-main.log 12 main 5433 online postgres /var/lib/postgresql/12/main /var/log/postgresql/postgresql-12-main.log pg_upgradecluster 10 main #This failed: pg_dump: error: query failed: ERROR: could not access file "$libdir/postgis-2.4": No such file or directory So... I can't automatically upgrade without first fixing the issue with v10 and postgis. add-apt-repository http://apt.postgresql.org/pub/repos/apt But that put the following into /etc/apt/sources-list: deb http://apt.postgresql.org/pub/repos/apt focal main vi it to (see https://wiki.postgresql.org/wiki/Apt): deb http://apt.postgresql.org/pub/repos/apt focal-pgdg wget --quiet -O - http://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | sudo apt-key add - apt-get update #Throws a warning: N: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'http://apt.postgresql.org/pub/repos/apt focal-pgdg InRelease' doesn't support architecture 'i386' But the wretched thing still doesn't seem to be available: apt-get install postgresql-10-postgis-2.4 Package postgresql-10-postgis-2.4 is not available, but is referred to by another package. Trying a manual approach. [https://packages.ubuntu.com/bionic/i386/postgresql-10-postgis-2.4/download Get the file], put it in /bulk/temp and cd there. Then: apt-get install ./postgresql-10-postgis-2.4_2.4.3+dfsg-4_i386.deb #This failed too - there are unmet dependencies and they are 'not installable'. ======Switching over the installations====== So, I took the alternative approach of changing the data folders [https://www.tutorialspoint.com/how-to-change-postgresql-data-folder-location-on-ubuntu-16-04]. The plan:#Take version 10 offline#Move version 10's data to a new location (/var/lib/postgresql/10/main)#Switch the ports of versions 10 and 12#Move version 12's data to /data#Put version 12 online#Load up the data in version 12!#Optionally wipe out the old installation Shut it down: pg_ctlcluster 12 master start pg_lsclusters #The cluster Ver Cluster Port Status Owner Data directory Log file 10 main 5432 online postgres /data/postgres /var/log/postgresql/postgresql-10-main.log 12 master 5433 online postgres /var/lib/postgresql/12/master /var/log/postgresql/postgresql-12-master.logg systemctl stop postgresql systemctl status postgresql Edit the config files: vi /etc/postgresql/10/main/postgresql.conf data_directory = '/var/lib/postgresql/10/main' port = 5433 vi /etc/postgresql/12/master/postgresql.conf data_directory = '/data/postgres' port = 5432 listen_addresses = '*' #While we are here do some performance tuning: shared_buffers = 512MB huge_pages = try temp_buffers = 8G work_mem = 4GB maintenance_work_mem = 64 effective_cache_size = 384GB #Note that I didn't reduce the number of connections (and the max_wal_senders, which must be < max connections), or change max_stack_depth (which gives an error if you set it too high) vi /etc/postgresql/12/master/pg_hba.conf Copy over the config to allow access from inside the network Move the data: df #to check diskspace rm -R /var/lib/postgresql/10/main #Note that none of the config files in here were valid (though you should check this is true before you do it!) rsync -av /data/postgres/ /var/lib/postgresql/10/main #Takes awhile, but make sure it is all done before the next step rm -R /data/postgres rsync -av /var/lib/postgresql/12/master/ /data/postgres systemctl start postgresql pg_lsclusters Ver Cluster Port Status Owner Data directory Log file 10 main 5433 online postgres /var/lib/postgresql/10/main /var/log/postgresql/postgresql-10-main.log 12 master 5432 online postgres /data/postgres /var/log/postgresql/postgresql-12-master.log Do the installs for some extensions: apt-get install postgresql-12-plr apt-get install postgresql-plperl-12 postgresql-plpython3-12 Check it all works: psql postgres CREATE EXTENSION postgis; CREATE EXTENSION plr; CREATE EXTENSION plperl; CREATE EXTENSION plpython3u; \dx List of installed extensions Name | Version | Schema | Description---------+---------+------------+--------------------------------------------------------------------- plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language postgis | 3.0.0 | public | PostGIS geometry, geography, and raster spatial types and functions (2 rows) Make the user: createuser --interactive researcher Then restore the databases (as researcher in /bulk/backup): createdb stockmarket pg_restore -Fc -d stockmarket stockmarket_Fc_20201023.dump createdb vcdb4 pg_restore -Fc -d vcdb4 vcdb4_Fc.dump The restore threw some errors related to not having extension plpythonu, but otherwise seemed fine. The issue seems to be that pythonu is python2[https://www.postgresql.org/docs/12/plpython-python23.html], and python2 is not available for postgres 12 (it might be here: https://wiki.postgresql.org/wiki/Apt): apt-cache search ".*plpython*.*" ====Other Fixes====

Remove redundant user accounts:

while [ 1 ] ; do echo -en "$(date +%T)\r" ; sleep 1; done

====Important Moves====

I kept the old versions of mediawiki and wordpress and moved them to /bulk/retired (using yyymmdd dates)

mv /var/www/html/blog20200809 /bulk/retired/

===Wordpress Redux===

====Install====

First, move the old folder to a new name, so that it is there for backup and then get the new install and unpack it.

</RequireAny>

====Set up====

Then set up the dbase by editing wp-config.php (it's easiest to modify the sample).

Ironically, it then recommends that I upgrade to PHP7.4... but that would just give issues for mediawiki. On the other hand, everything is now green and just 4 groups of recommendations remain.

====Config====

See [[Wordpress Blog Site (Tool)]] for the McNair Center's build.

Using www.edegan.com/blog/wp-admin I configured the blog as follows:

*Select Twenty Twenty as the theme

*Add the permalink code to the .htaccess file, so that the URLs will work with postnames

*Copy over images to wp-content/uploads (use cp -a to maintain permissions)

*Change the site name to https (after fixing the https setup, see below)

Install plugins:

*Wordfence Security

*Disable Comments

*Site Kit by Google(set up once live!)

*Pixabay

I also added:

*CoBlocks (free)

*Advanced Gutenberg (free)

*Otter

I didn't add Co-Authors Plus (https://wordpress.org/plugins/co-authors-plus/) as it hasn't been tested on the latest version of wordpress. There are other plugins that offer equivalent functionality if I need one later.

Other plugins I might want are:

*Revive Old Post (share with twitter)

*Optimole (optimize images)

*WP Rocket (implement cache)

Notes:

*Twitter embedding: https://www.wpbeginner.com/wp-tutorials/how-to-display-recent-tweets-in-wordpress-with-twitter-widgets/

====Hardening Wordpress====

I hardened the wordpress installation: https://wordpress.org/support/article/hardening-wordpress/

This included:

*Fixing file ownership: For fully hardened, change ownership of everything to root, except wflogs, uploads and themes in wp-content, which should be owned by www-data. However, then you won't be able to install plugins etc. A compromise is -R root:root for blog and then www-data:www-data for wp-content.

*Check file permissions: Everything is 644, except wp-content which is 755

*Checking dbase rights and setting new passwords.

*Changing passwords on old accounts (with posts, so the accounts shouldn't be deleted) to random strong strings.

*Fixing up .htaccess file to impose restrictions

*Install Sucuri

*Enable more logging

Checking user rights in the dbase and changing their password:

mysql -user=root -p

use wordpress

SELECT User FROM mysql.user;

SHOW GRANTS FOR 'username'@'localhost';

SET PASSWORD FOR 'username'@'localhost'='newpassword';

(Note that this shouldn't be logged in clear on the server, but might be on a client. Delete .mysql_history at the end of your session.)

.htaccess in wp-includes:

# Block the include-only files.

RewriteEngine On

RewriteBase /

RewriteRule ^wp-admin/includes/ - [F,L]

RewriteRule !^wp-includes/ - [S=3]

RewriteRule ^wp-includes/[^/]+\.php$ - [F,L]

RewriteRule ^wp-includes/js/tinymce/langs/.+\.php - [F,L]

RewriteRule ^wp-includes/theme-compat/ - [F,L]

</IfModule>

# BEGIN WordPress

The #BEGIN WordPress tag is redundant as the file is 644 root.

Add the following to .htaccess in the wordpress dir:

order allow,deny

deny from all

</files>

If there are plugin installation issues then add to wp-config.php

define('FS_METHOD','direct');

Once I'm all done with the theme etc., I can uncomment the following from wp-config.php

define('DISALLOW_FILE_EDIT', true);

{{Colored box|title=Notice|content=With hardened permissions, you won't be able to update Wordpress from the dashboard. To fix this, set ownership of the entire wordpress directory to www-data:www-data (i.e., chown -R www-data:www-data blog/), run the update, and then revert the ownership to root (or some other account).}}

====Redesign====

I built a [[Branding]] palette, to standardize the colors. And I installed the Twentig plugin, to give extra configuration options.

I changed the site colors, added the logo and the tag line, and made other config changes.

Then, I added custom CSS as follows.

To reduce the header spacing:

.header-inner {

padding: 1.5rem 0;

}

To remove the title from the landing page:

.page-id-2169 .entry-title{

display:none !important;

}

.page-id-2169 .entry-header {

padding: 0;

}

To do:

*~~Get~~ I need to add social media icons! That might be as easy as adding the social media ~~to work?~~menu [https://wordpress.org/support/article/twenty-twenty/#add-social-icons].*Get ~~HTTPS working~~a '''related posts widget'''? There's Yet Another Related Posts Plugin, Contextual Related Posts, and Inline Related Posts... I went with YARPP, as it is the most popular. It is apparently resource-heavy.

I tried the following blocks plugins:

*'''Ultimate Addons for Guttenberg'''

*It's free and adds some nice basic functionality

**Post blocks include: Post Carousel, Post Grid, Post Masonry, Post Timeline, Advanced Columns -- but customization is limited and I can't do one post

*'''Getwid''':

**Pretty highly customizable.

**Can specify which posts to show in 3 blocks (Custom Post Type, Post Carousel, and Post Slider) and can build custom templates to arrange how they are displayed

**Post blocks: Recent Posts, Custom Post Type, Post Carousel, and Post Slider

*'''Redux''' -- It's a templates library. You get 5 for free and they upsell hard.

*'''ZeGuten''' - Couldn't find it

*'''Advanced Gutenberg''' - It's free and widely used...

*'''CoBlocks''' -- Does the basics

**Posts -- Can't specify specific posts. Can do category.

**Post Carousel -- Likewise.

*'''Stackable''':

**It requested opt-in, which I didn't like, and it wants you to 'Go Premium'.

**It has settings for everything! By far the most detailed configuration.

**Useful blocks:

***Posts -- can't seem to specify a specific post

***Advanced Columns and Grids -- for layout

***Card -- could make posts links with buttons

***Feature/Feature Grid -- likewise

***Container? Might be helpful

*'''Gutenberg Post Blocks'''

**Untested with my version. Seems to work.

**Has lots of options but does full-page things. Can limit to a post using include but has next page links...

**Tried to push for an update to pro.

*'''Magical Posts Display''' -- I dumped it for being too weird.

*'''Otter Blocks'''

**Google maps block and other useful things... I just don't need it right now.

~~Twitter Widget~~Built-in:*Latest Posts (widget)

~~===Other things===~~Chosen block plugins:*'''Getwid''' -- It's outstanding and embraces templates for serious bespoke customization*'''Stackable''' -- For its option-based customization*I might add back '''coblocks''', '''Advanced Gutenberg''' and '''Ultimate Addons for Gutenberg'''

~~I should harden the wordpress installation: https://wordpress.org/support/article/hardening-wordpress/~~====Email====

I installed WP Mail SMTP Lite. I first set it up to use Google. Essentially you need to sign in to Google and set up an API in the console: https://console.developers.google.com/flows/enableapi?apiid=gmail&pli=1. However, this seemed to introduce a massive security hole unless you have G Suite, so I abandoned this approach. I had previously set up SMTP through Google for the wiki (See [[Research_Computing_Configuration#Confirm_Account]]). So, I used the same approach with Wordpress. In WP Mail SMTP Lite choose 'Other' (see the [https://www.wpbeginner.com/plugins/how-to-send-email-in-wordpress-using-the-gmail-smtp-server/ second method]). Then [https://wpmailsmtp.com/docs/how-to-secure-smtp-settings-by-using-constants/ edit wp-config.php to hardcode the values] (this ensures that the password, which is stored plain-text, is a little more secure.): define( 'WPMS_ON', true ); //You MUST set this if you want hardcoded values to work! define( 'WPMS_LICENSE_KEY', '' ); define( 'WPMS_MAIL_FROM', 'blog@edegan.com' ); define( 'WPMS_MAIL_FROM_FORCE', true ); define( 'WPMS_MAIL_FROM_NAME', 'The Blog at EdEgan.com' ); define( 'WPMS_MAIL_FROM_NAME_FORCE', true ); define( 'WPMS_MAILER', 'smtp' ); // Possible values: 'mail', 'gmail', 'mailgun', 'sendgrid', 'smtp'. define( 'WPMS_SET_RETURN_PATH', true ); define( 'WPMS_SMTP_HOST', 'ssl://smtp.gmail.com' ); define( 'WPMS_SMTP_PORT', 465 ); define( 'WPMS_SSL', 'ssl' ); // Possible values '', 'ssl', 'tls' - note TLS is not STARTTLS. define( 'WPMS_SMTP_AUTH', true ); define( 'WPMS_SMTP_USER', 'username@gmail.com' ); // SMTP authentication username, only used if WPMS_SMTP_AUTH is true. define( 'WPMS_SMTP_PASS', 'password generated by Google' ); define( 'WPMS_SMTP_AUTOTLS', true ); =====Author Comments===== The blog supports multiple authors and by default, Wordpress emails an author whenever one of their posts gets a comment. If you'd like to disable author comment emails but keep the moderator emails, there's a simple fix: '''Just go to wp-admin/options.php and set 'comments_notify' to 0.''' (See https://codex.wordpress.org/Option_Reference) More complicated methods involve writing your own plugin [https://wordpress.stackexchange.com/questions/150125/disabling-comment-notifications-for-post-author] to refine wp_new_comment_notify_postauthor[https://developer.wordpress.org/reference/functions/wp_new_comment_notify_postauthor/] or changing the hooks[https://developer.wordpress.org/reference/hooks/notify_post_author/] used in wp-includes/comment.php: $maybe_notify = apply_filters( 'notify_post_author', $maybe_notify, $comment_ID ); ====Social Media Integration==== Getting the social media icons on the menu and correctly linked up is very straight forward. Follow the [https://wordpress.org/support/article/twenty-fifteen/#add-social-icons guide for twenty-fifteen], which also works for 2020. Getting some share buttons was more problematic, particular as my planned social media usage is somewhat atypical (Twitter, LinkedIn, and Reddit, really in reverse order), and because I don't want to pay anything. The free version of [https://revive.social/plugins/revive-old-post/ Revive Old Posts] lets you push content to Twitter and Facebook, but they want you pay to push to LinkedIn. The best free options seem to be:*[https://wordpress.org/plugins/add-to-any/ AddToAny Share Buttons] - Integrates with Google Analytics*[https://wordpress.org/plugins/simple-social-icons/ Simple Social Icons] - The simplest option*[https://wordpress.org/plugins/shared-counts/ Shared Counts] -- Counts hits (but using a 3rd party for data?)*[https://wordpress.org/plugins/wordpress-social-login/ WordPress Social Login] - if you want users to log in using their SM accounts (note: has a bimodal ratings distro)*[https://wordpress.org/plugins/jetpack/ JetPack] -- The plugin used by wordpress.com for this functionality. The free version should suffice, but this thing is a monster. It also uses an account on the wordpress.com cloud, which is a pain for those who are self-hosting. I went with AddToAny, as it had the most installations, is entirely open-source, and offers all the functionality I need. ====Avoiding JetPack==== I tried to add a profile picture, but by default, WordPress uses [https://en.gravatar.com/ Gravitar], which, surprise, surprise, links to your WordPress.com account... and to add a self-hosted site, you have to install JetPack. At this point, I felt harassed and doubly so because I didn't install JetPack and yet, some how, the profile picture correctly updated from the one I'd posted on Gravitar. What the hecK? ===SEO=== I used Site Kit plugin for wordpess, and for mediawiki I made a sitemap to submit to Google. See https://www.mediawiki.org/wiki/Manual:GenerateSitemap.php In mediawiki: mkdir sitemap php maintenance/generateSitemap.php --memory-limit=50M --fspath=/var/www/html/mediawiki/sitemap/ --identifier=edegancom --urlpath=/sitemap/ --server=https://www.edegan.com --compress=yes Then submit it to Google... I did this by making an alias in apache2.conf from sitemap to /var/www/html/mediawiki/sitemap/, then submitting https://www.edegan.com/sitemap/sitemap-index-edegancom.xml #in retrospect, I wish I'd used an identifier with 'wiki' in it but what the hey. And with that success behind you, install Google XML Sitemaps on Wordpress, chose some settings (on Settings -> XML-Sitemap), and then post the URL to Google: https://www.edegan.com/blog/sitemap.xml It seems Yoast already builds a sitemap, you just need to submit to it Google... (I uninstalled XML Sitemaps): https://www.edegan.com/blog/sitemap_index.xml ===HTTPS===

To set up HTTPS using Let's Encrypt, see https://linuxize.com/post/secure-apache-with-let-s-encrypt-on-ubuntu-20-04/

Install it and make some directories...

apt update

apt install certbot

openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

takes ~20 secs

mkdir -p /var/lib/letsencrypt/.well-known

chgrp www-data /var/lib/letsencrypt

chmod g+s /var/lib/letsencrypt

Set up the config files

vi /etc/apache2/conf-available/letsencrypt.conf

Alias /.well-known/acme-challenge/ "/var/lib/letsencrypt/.well-known/acme-challenge/"

AllowOverride None

Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec

Require method GET POST OPTIONS

</Directory>

vi /etc/apache2/conf-available/ssl-params.conf

SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1

SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA- CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384

SSLHonorCipherOrder off

SSLSessionTickets off

SSLUseStapling On

SSLStaplingCache "shmcb:logs/ssl_stapling(32768)"

SSLOpenSSLConfCmd DHParameters "/etc/ssl/certs/dhparam.pem"

Header always set Strict-Transport-Security "max-age=63072000"

Enable some apache2 mods!

a2enmod ssl

a2enmod headers

a2enconf letsencrypt

a2enconf ssl-params

sudo a2enmod http2

systemctl reload apache2

Run certbot!

certbot certonly --agree-tos --email ed@edegan.com --webroot -w /var/lib/letsencrypt/ -d edegan.com -d www.edegan.com

Note that I needed an @ entry in my A record for edegan.com pointed to my IP address to get the main challenge to succeed.

Then set up a new apache2 config file (in /etc/apache):

mv sites-available/000-default.conf sites-available/000-default.conf.bak

vi sites-available/edegan.com.conf

ServerName www.edegan.com

ServerAdmin ed@edegan.com

Redirect permanent / https://www.edegan.com/

</VirtualHost>

ServerName www.edegan.com

Protocols h2 http/1.1

DocumentRoot /var/www/html

ErrorLog ${APACHE_LOG_DIR}/error.log

CustomLog ${APACHE_LOG_DIR}/access.log combined

SSLEngine On

SSLCertificateFile /etc/letsencrypt/live/edegan.com/fullchain.pem

SSLCertificateKeyFile /etc/letsencrypt/live/edegan.com/privkey.pem

# Other Apache Configuration

Alias /wiki /var/www/html/mediawiki/index.php

RewriteEngine On

RewriteRule ^/*$ %{DOCUMENT_ROOT}/mediawiki/index.php [L]

</VirtualHost>

ln -s sites-available/edegan.com.conf sites-enabled/edegan.com.conf

systemctl reload apache2

Test it by going to https://www.ssllabs.com/ssltest/

Finally, edit /etc/cron.d/certbot and append the following to the last line (after -renew):

--renew-hook "systemctl reload apache2"

certbot renew --dry-run

Tests the renewal!

====PDFEmbed Issue====

Enabling and requiring HTTPS causes an issue with PDFEmbed on mediawiki, where you get a blank frame. The PDF is still there, other images load fine, but the PDF frame won't render the PDF. The problem is actually that the PDF is served with HTTP and the rest of the page is served with HTTPS, and Chrome (and perhaps other browsers) don't render the insecure content as a consequence (see [https://www.mediawiki.org/wiki/Topic:Uhgnq0wbmzfurbj0] for a description of the symptoms, but not the solution.).

The solution is to edit mediawiki/extensions/PDFEmbed/PDFEmbed.hooks.php. For me it was line 103 that previously said:

'src' => $file->getFullUrl().'#page='.$page,

I changed this line to:

'src' => preg_replace("/^http:/i", "https:", $file->getFullUrl()).'#page='.$page,

This is mentioned in a comment on a topic page, though presumably for an earlier version: https://www.mediawiki.org/wiki/Topic:Syxow0why4c0cvvm

====Another Issue====

Interestingly, I started getting a message from Google Chrome whenever I went to post wiki entries saying:

"The information you’re about to submit is not secure". There's an option to "Proceed anyway" or "Go back".

This started after I had MultiTail running viewing apache's logs, but I couldn't see, beyond some kind of file lock examination, how it could work. I figured that it was a coincidence and something else might have happened.

My first thought was that my SSL certificate might have expired. However, the certificate looks valid and good, and the issue survived a reboot.

By inspecting the webpages (in Chrome) and then reviewing the Console, I could see that it was caused by a mixed content problem:

Mixed Content: The page at '<URL>' was loaded over HTTPS, but requested an insecure font '<URL>'.

This request has been blocked; the content must be served over HTTPS.

It seemed that I somehow have some font addresses hardcoded somewhere:

Mixed Content: The page at ... was loaded over HTTPS, but requested an insecure font

'http://128.42.44.180/mediawiki/resources/assets/fonts/BonvenoCF-Light.otf'.

This request has been blocked; the content must be served over HTTPS.

The copy of Chrome on my desktop must somehow have been upgraded? Or something else changed to cause a change in behavior...

The IP is from the old web server at the McNair Center, suggesting that when I migrated the McNair database into the new wiki, I migrated this issue. (Note that it doesn't appear to be something hardcoded into a .css file, or similar -- I can't find any trace on the filesystem and besides, this wiki was built from a fresh install.)

I found the URLs hardcoded in [[MediaWiki:Common.css]] (it must have been moved with the last big batch of pages and I somehow didn't notice!) but then couldn't edit it! It seems that following [https://www.mediawiki.org/wiki/MediaWiki_1.32/interface-admin Mediawiki 1.32], the rights to edit the interface were separated out, and users now need the editinterface right to change anything in the Mediawiki namespace. So, I went to [[Special:UserRights]] and gave myself permission. Then I edited the page, which changed the look-and-feel of my editor (I have no idea why), removed the consol messages, but left the problem (even after ctrl-shift-r cache flush on Chrome).

===Install VSFTPD===

With the security restrictions on wordpress, I now need an FTP server to get files for themes, plugins, etc. I like VSFTPD, as its simple, secure, and has a nice standalone config. Old documentation on an earlier install on the old [[Wordpress Blog Site (Tool)]] page. Instructions are here: https://linuxconfig.org/how-to-setup-ftp-server-on-ubuntu-20-04-focal-fossa-linux

apt-get install vsftpd

cp /etc/vsftpd.conf /etc/vsftpd.conf_orig

vi /etc/vsftpd.conf

#Change the following

write_enable=YES

local_umask=022

ssl_enable=YES

#Add the following (forces ssl)

allow_anon_ssl=NO

force_local_data_ssl=YES

force_local_logins_ssl=YES

ssl_tlsv1=YES

ssl_sslv2=YES

ssl_sslv3=YES

/etc/init.d/vsftpd restart

Then add a user and set it up:

useradd -m blog

passwd blog

usermod -a -G www-data blog

usermod -d /var/www/html/blog blog

Test it:

ftp 127.0.0.1

sftp 127.0.0.1

Changes

Research Computing Configuration (view source)

Revision as of 14:10, 5 February 2024

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Sites

Sections

Organizations

Help

Tools