A blog and website by Peter Bengtsson
09 April 2020 0 comments Linux, Nginx
I knew I didn't want to download the sources to
nginx to install it on my new Ubuntu 19.10 server because I'll never have the discipline to remember to keep it upgraded. No, I'd rather just run
apt update && apt upgrade every now and then.
Why is this so hard?! All I need is the ability to set
brotli_static on; in my Nginx config so it'll automatically pick the
.br file if it exists on disk.
These instructions totally helped but here they are specifically for my version (all run as
git clone --recursive https://github.com/google/ngx_brotli.git
apt install brotli
apt-get build-dep nginx
# Note the version of which nginx you have installed
# ...which informs which URL to wget
nginx -V 2>&1 >/dev/null | grep -o " --.*" | grep -oP .+?(?=--add-dynamic-module)| head -1 > nginx-1.17.9/build_args.txt
./configure --with-compat $(cat build_args.txt) --add-dynamic-module=../ngx_brotli
cp objs/ngx_http_brotli_filter_module.so /usr/lib/nginx/modules/
chmod 644 /usr/lib/nginx/modules/ngx_http_brotli_filter_module.so
cp objs/ngx_http_brotli_static_module.so /usr/lib/nginx/modules/
chmod 644 /usr/lib/nginx/modules/ngx_http_brotli_static_module.so
ls -l /etc/nginx/modules
Now I can edit my
/etc/nginx/nginx.conf (somewhere near the top) to:
And test that it works:
08 April 2020 0 comments Linux, Node
I'm setting up a new Ubuntu (Eoan Ermine) 19.10 server and I noticed that
apt install nodejs gives you Node v10 which is an LTS (Long Term Support) version that'll last till April 2021
. However, I want Node v12 which is the most recent LTS release as of April 2020.
To install it I used these instructions:
curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
sudo apt-get install -y nodejs
That worked great.
When it finished, it spat out this nice little blurb about how to install
Fetched 7454 B in 1s (12.3 kB/s)
Reading package lists... Done
## Run `sudo apt-get install -y nodejs` to install Node.js 12.x and npm
## You may also need development tools to build native addons:
sudo apt-get install gcc g++ make
## To install the Yarn package manager, run:
curl -sL https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list
sudo apt-get update && sudo apt-get install yarn
By the way, I have no idea what
nodejs-mozilla but running
apt show nodejs-mozilla yields:
Maintainer: Ubuntu Developers <email@example.com>
Installed-Size: 42.0 MB
Depends: libc6 (>= 2.29), libgcc1 (>= 1:3.4), libstdc++6 (>= 9)
Download-Size: 10.4 MB
APT-Sources: http://mirrors.digitalocean.com/ubuntu eoan-updates/universe amd64 Packages
building fast, scalable network applications. Node.js uses an
event-driven, non-blocking I/O model that makes it lightweight and
efficient, perfect for data-intensive real-time applications that run
across distributed devices.
Node.js is bundled with several useful libraries to handle server
System, Events, Standard I/O, Modules, Timers, Child Processes, POSIX,
HTTP, Multipart Parsing, TCP, DNS, Assert, Path, URL, Query Strings.
Installing it doesn't add a
node executable and I can't find a home page for it.
apt can be weird sometimes.
19 February 2020 0 comments Python, MacOSX
I'm still working on getting
pyenv in my bloodstream. It seems like totally the right tool for having different versions of Python available on macOS that don't suddenly break when you run
periodically. But every thing I tried failed with an error similar to this:
python-build: use openssl from homebrew
python-build: use readline from homebrew
python-build: use readline from homebrew
BUILD FAILED (OS X 10.15.x using python-build 20XXXXXX)
Inspect or clean up the working tree at /var/folders/mw/0ddksqyn4x18lbwftnc5dg0w0000gn/T/python-build.20190528163135.60751
Results logged to /var/folders/mw/0ddksqyn4x18lbwftnc5dg0w0000gn/T/python-build.20190528163135.60751.log
Last 10 log lines:
./Modules/posixmodule.c:5924:9: warning: this function declaration is not a prototype [-Wstrict-prototypes]
if (openpty(&master_fd, &slave_fd, NULL, NULL, NULL) != 0)
./Modules/posixmodule.c:6018:11: error: implicit declaration of function 'forkpty' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
pid = forkpty(&master_fd, NULL, NULL, NULL);
./Modules/posixmodule.c:6018:11: warning: this function declaration is not a prototype [-Wstrict-prototypes]
2 warnings and 2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
I read through the Troubleshooting FAQ and the "Common build problems" documentation.
xcode was up to date and I had all the related
brew packages upgraded. Nothing seemed to work.
Until I saw this comment on an open
pyenv issue: "Unable to install any Python version on MacOS"
All I had to do was replace the
10.15 and now it finally worked here on Catalina 10.15. So, the magical line was this:
pyenv install -v 3.7.6
Hopefully, by blogging about it you'll find this from Googling and I'll remember the next time I need it because it did eat 2 hours of precious evening coding time.
14 February 2020 0 comments Python
You can accomplish the same with
curl -L but I've had this as a little personal hack script in my
~/bin folder on my computer. Thought I'd make it a public tool. Also, from here, a lot more can be done to this script if you wanna help out with ideas.
▶ redirect-chain http://developer.mozilla.org/en-US/docs/xpcshell
0 http://developer.mozilla.org/en-US/docs/xpcshell 301
1 > https://developer.mozilla.org/en-US/docs/xpcshell 301
2 >> https://developer.mozilla.org/docs/en/XPConnect/xpcshell 302
3 >>> https://developer.mozilla.org/en-US/docs/en/XPConnect/xpcshell 301
4 >>>> https://developer.mozilla.org/en-US/docs/XPConnect/xpcshell 301
5 >>>>> https://developer.mozilla.org/en-US/docs/Mozilla/XPConnect/xpcshell 301
6 >>>>>> https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Language_bindings/XPConnect/xpcshell 200
It basically gives you a pretty summary of redirects from a starting URL.
To install it on your system run:
pipx install redirect-chain
07 February 2020 0 comments Python
We use poetry in MDN Kuma. That means there's a
pyproject.toml and a
poetry.lock file. To add or remove dependencies, you don't touch either file in an editor. For example, to add a package:
poetry add --dev black
poetry.lock for you. (Same with
yarn add somelib which edits
Suppose that you make a pull request to add a new dependency, but someone sneaks a new pull request in before you and have theirs landed in
master before. Well, that's how you end up in this place:
So how do you resolve that?
So, you go back to your branch and run something like:
git checkout master
git pull origin master
git checkout my-branch
git merge master
Now you get this in
(use "git add <file>..." to mark resolution)
both modified: poetry.lock
And the contents of
poetry.lock looks something like this:
I wish there just was a way
poetry itself could just figure fix this.
What you need to do is to run:
# Get poetry.lock to look like it does in master
git checkout --theirs poetry.lock
# Rewrite the lock file
poetry.lock file should correctly reflect the
pyproject.toml that has been merged from
To finish up, resolve the conflict:
git add poetry.lock
git commit -a -m "conflict resolved"
# and most likely needed
poetry.lock file there's the lock file's hash. It looks like this:
content-hash = "875b6a3628489658b323851ce6fe8dafacd5f69e5150d8bb92b8c53da954c1be"
So, as can be seen in my screenshot, when git conflicted on this it looks like this:
+content-hash = "6658b1379d6153dd603bbc27d04668e5e93068212c50e76bd068e9f10c0bec59"
content-hash = "5c00dce18ddffd5d6f797dfa14e4d56bf32bbc3769d7b761a2b1b3ff14bce287"
content-hash = "5c00dce1... is what you'd find in
content-hash = "6658b137... is what you would see in your branch before the conflict.
When you run that
poetry lock you can validate that the new locking worked because it should be a hash. One that is neither
I'm still new to
poetry and I'm learning. This was just some loud note-to-self so I can remember for next time.
I don't yet know what else can be automated if there's a conflict in
pyproject.toml too. And what do you do if there are serious underlying conflicts in Python packages, like they added a package that requires
somelib<=0.99 and you added something that requires
Also, perhaps there are ongoing efforts within the
poetry project to help out with this.
UPDATE Feb 12, 2020
My colleague informed me that this change was actually NOT what I wanted.
poetry lock actually updates some dependencies as it makes a completely new lock file. I didn't immediately notice that in my case because the lock file is large. See this open issue which is about the ability to update the lock file without upgrading any other dependencies.
05 February 2020 0 comments Python, MacOSX
I don't know how many times I've encountered this but by blogging about it, hopefully, next time it'll help me, and you!, find this sooner.
If you get this:
clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/peterbe/.pyenv/versions/3.8.0/lib -L/opt/boxen/homebrew/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/Users/peterbe/.pyenv/versions/3.8.0/lib -L/opt/boxen/homebrew/lib -L/opt/boxen/homebrew/lib -I/opt/boxen/homebrew/include build/temp.macosx-10.14-x86_64-3.8/MySQLdb/_mysql.o -L/usr/local/Cellar/mysql/8.0.18_1/lib -lmysqlclient -lssl -lcrypto -o build/lib.macosx-10.14-x86_64-3.8/MySQLdb/_mysql.cpython-38-darwin.so
ld: library not found for -lssl
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'clang' failed with exit status 1
(The most important line is the
ld: library not found for -lssl)
On most macOS systems, when trying to install a Python package that requires a binary compile step based on the system
(which I think comes from the OS), you'll get this.
The solution is simple, run this first:
Depending on your install of things, you might need to adjust this accordingly. For me, I have:
▶ ls -l /usr/local/opt/openssl/
-rw-r--r-- 1 peterbe staff 717 Sep 10 09:13 AUTHORS
-rw-r--r-- 1 peterbe staff 582924 Dec 19 11:32 CHANGES
-rw-r--r-- 1 peterbe staff 743 Dec 19 11:32 INSTALL_RECEIPT.json
-rw-r--r-- 1 peterbe staff 6121 Sep 10 09:13 LICENSE
-rw-r--r-- 1 peterbe staff 42183 Sep 10 09:13 NEWS
-rw-r--r-- 1 peterbe staff 3158 Sep 10 09:13 README
drwxr-xr-x 4 peterbe staff 128 Dec 19 11:32 bin
drwxr-xr-x 3 peterbe staff 96 Sep 10 09:13 include
drwxr-xr-x 10 peterbe staff 320 Sep 10 09:13 lib
drwxr-xr-x 4 peterbe staff 128 Sep 10 09:13 share
Now, with those things set you should hopefully be able to do things like:
pip install mysqlclient