Peterbe.com

A blog and website by Peter Bengtsson

Sort a JavaScript array by some boolean operation

02 December 2021 0 comments   JavaScript


Imagine you have an array like this:

const items = [
  { num: 'one', labels: [] },
  { num: 'two', labels: ['foo'] },
  { num: 'three', labels: ['bar'] },
  { num: 'four', labels: ['foo'] },
  { num: 'five', labels: [] },
];

What you want, is to sort them in a way that all those entries that have a label foo come first, but you don't want to "disturb" the existing order. Essentially you want this to be the end result:

{ num: 'two', labels: ['foo'] },
{ num: 'four', labels: ['foo'] },

{ num: 'one', labels: [] },
{ num: 'three', labels: ['bar'] },
{ num: 'five', labels: [] },

Here's a way to do that:

items.sort(
  (itemA, itemB) =>
    Number(itemB.labels.includes('foo')) - Number(itemA.labels.includes('foo'))
);
console.log(items);

And the outcome is:

[
  { num: 'two', labels: [ 'foo' ] },
  { num: 'four', labels: [ 'foo' ] },
  { num: 'one', labels: [] },
  { num: 'three', labels: [ 'bar' ] },
  { num: 'five', labels: [] }
]

The simple trick is to turn then test operation into a number (0 or 1) and you can do that with Number.

Brotli compression quality comparison in the real world

01 December 2021 1 comment   Node, JavaScript


At work, we use Brotli (using the Node builtin zlib) to compress these large .json files to .json.br files. When using zlib.brotliCompress you can set options to override the quality number. Here's an example of it at quality 6:

import { promisify } from 'util'
import zlib from 'zlib'
const brotliCompress = promisify(zlib.brotliCompress)

const options = {
  params: {
    [zlib.constants.BROTLI_PARAM_MODE]: zlib.constants.BROTLI_MODE_TEXT,
    [zlib.constants.BROTLI_PARAM_QUALITY]: 6,
  },
}

export async function compress(data) {
  return brotliCompress(data, options)
}

But what if you mess with that number. Surely, the files will become smaller, but at what cost? Well, I wrote a Node script that measured how long it would take to compress 6 large (~25MB each) .json file synchronously. Then, I put them into a Google spreadsheet and voila:

Size

Total size per level

Time

Total seconds per level

Miles away from rocket science but I thought it was cute to visualize as a way of understanding the quality option.

How to string pad a string in Python with a variable

19 October 2021 1 comment   Python


I just have to write this down because that's the rule; if I find myself googling something basic like this more than once, it's worth blogging about.

Suppose you have a string and you want to pad with empty spaces. You have 2 options:

>>> s = "peter"
>>> s.ljust(10)
'peter     '
>>> f"{s:<10}"
'peter     '

The f-string notation is often more convenient because it can be combined with other formatting directives.
But, suppose the number 10 isn't hardcoded like that. Suppose it's a variable:

>>> s = "peter"
>>> width = 11
>>> s.ljust(width)
'peter      '
>>> f"{s:<width}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid format specifier

Well, the way you need to do it with f-string formatting, when it's a variable like that is this syntax:

>>> f"{s:<{width}}"
'peter      '

How to bulk-insert Firestore documents in a Firebase Cloud function

23 September 2021 0 comments   Node, Firebase, JavaScript


You can't batch-add/bulk-insert documents in the Firebase Web SDK. But you can with the Firebase Admin Node SDK. Like, in a Firebase Cloud Function. Here's an example of how to do that:

const firestore = admin.firestore();
let batch = firestore.batch();
let counter = 0;
let totalCounter = 0;
const promises = [];
for (const thing of MANY_MANY_THINGS) {
  counter++;
  const docRef = firestore.collection("MY_COLLECTION").doc();
  batch.set(docRef, {
    foo: thing.foo,
    bar: thing.bar,
    favNumber: 0,
  });
  counter++;
  if (counter >= 500) {
    console.log(`Committing batch of ${counter}`);
    promises.push(batch.commit());
    totalCounter += counter;
    counter = 0;
    batch = firestore.batch();
  }
}
if (counter) {
  console.log(`Committing batch of ${counter}`);
  promises.push(batch.commit());
  totalCounter += counter;
}
await Promise.all(promises);
console.log(`Committed total of ${totalCounter}`);

I'm using this in a Cloud HTTP function where I can submit a large amount of data and have each one fill up a collection.

I'm a GitHubber now

21 September 2021 1 comment   Work, GitHub


Starting today, I'm a Hubber. Meaning, I work for GitHub. I'll be joining the GitHub Docs team to help technical writers document all the various products that GitHub have. Since I haven't actually started coding anything yet, I don't want to claim I know how it works or exactly what I'll be working but on, but I do know that the site at hand is docs.github.com and I've previously taken a lot of inspiration from this site when building the MDN rewrite.

GitHub profile

If you are a Hubber, too, and reading this; Hi! Let's be friends! I'm a friendly guy. Please ping me and say hi.
I'll be working from home here in Mount Pleasant, South Carolina. I have 2 young kids; Tucker and Charlotte, who mean the world to me. And my backbone-of-life wife Ashley. My hobbies are coding, swimming, golf, and cooking.

TypeScript generic async function wrapper function

12 September 2021 0 comments   JavaScript


I find this so fiddly! I love TypeScript and will continue to use it if there's a choice. But I just wanted to write a simple async function wrapper and I had to Google for it and nothing was quite right. Here's my simple solution, as an example:

function wrappedAsyncFunction<T>(
    fn: (...args: any[]) => Promise<T>
  ): (...args: any[]) => Promise<T> {
    return async function(...args: any[]) {
      console.time("Took");
      try {
        return await fn(...args);
      } catch(error) {
        console.warn("FYI, an error happened:", error);
        throw error;
      } finally {
        console.timeEnd("Took");
      }

    };
  }

Here's a Playground demo

What I use it for is to wrap my Firebase Cloud Functions so that if any error happens, I can send that error to Rollbar. In particular, here's an example of it in use:

diff --git a/functions/src/cleanup-thumbnails.ts b/functions/src/cleanup-thumbnails.ts
index 46bdb34..a3e8d54 100644
--- a/functions/src/cleanup-thumbnails.ts
+++ b/functions/src/cleanup-thumbnails.ts
@@ -2,6 +2,8 @@ import * as admin from "firebase-admin";
 import * as functions from "firebase-functions";
 import { logger } from "firebase-functions";

+import { wrappedLogError } from "./rollbar-logger";
+
 const OLD_DAYS = 30 * 6; // 6 months
 // const ADDITIONAL_DAYS_BACK = 5;
 // const ADDITIONAL_DAYS_BACK = 15;
@@ -9,7 +11,7 @@ const PREFIX = "thumbnails";

 export const scheduledCleanupThumbnails = functions.pubsub
   .schedule("every 24 hours")
-  .onRun(async () => {
+  .onRun(wrappedLogError(async () => {
     logger.debug("Running scheduledCleanupThumbnails");

...

And my wrappedLogError looks like this:

export function wrappedLogError<T>(
  fn: (...args: any[]) => Promise<T>
): (...args: any[]) => Promise<T> {
  return async function(...args: any[]) {
    try {
      return await fn(...args);
    } catch (error) {
      logError(error);
      throw error;
    }
  };
}

I'm not sure it's the best or correct way to do it, but it seems to work. Perhaps there's a more correct solution but for now I'll ship this because it seems to work fine.