Angular Conf Melbourne 2018: Conference Recap

It was early morning rush hour on Friday, June 22nd, and the Melbourne winter sun shone on my face. I was attending Angular Conf Australia, being held at South Wharf. Arriving with just enough time to spare, I looked around the conference venue and had a quick chat with some of the participants.

After a slight delay, the conference kicked-off with a quick thanks to sponsors and supporters. In this post I’m going to give you a rundown of what I saw next.

An Introduction to Angular Progressive Web Applications

A couple of weeks ago I attended Angular Conf Australia and did a full-day workshop on Progressive Web Applications. In this post I’ll talk about what I learned. In doing so, I’ll show the basic concepts of progressive web applications and service workers, and demonstrate how service workers work in a real application.

What the heck is {N}ativescript?

ns

Native-what?

If you’re anything like myself before embarking on this project then you might wonder the same thing. Everyone has heard of React Native, it’s a popular technology enabling a cross-platform mobile development experience, but not so many have crossed paths with Nativescript, or {N}.

Nativescript is a framework with a similar goal to React Native, providing the ability to publish native apps for both major platforms, iOS and Android, whilst only maintaining a single (mostly) JavaScript codebase.  It is available in three flavours: standard, Angular, and Vue.js, allowing developers with experience in the latter two frameworks to use their existing skills whilst building for mobile.

Put simply, it’s a JavaScript runtime inside a native app, translating code on the fly into native elements.  In this post, I’m going to tell you about a few of the nice experiences we’ve had recently developing with Nativescript, as well as a few of the not so nice issues that were encountered, and whether we’d use it again.

Introducing column based partitioning in BigQuery

Some background

When we started using Google BigQuery – almost five years ago now – it didn’t have any partitioning functionality built into it.  Heck, queries cost $20 p/TB back then too for goodness’ sake!  To compensate for this lack of functionality and to save costs, we had to manually shard our tables using the well known _YYYYMMDD suffix pattern just like everyone else.  This works fine, but it’s quite cumbersome, has some hard limits, and your SQL can quickly becomes unruly.

Then about a year ago, the BigQuery team released ingestion time partitioning.  This allowed users to partition tables based on the load/arrival time of the data, or by explicitly stating the partition to load the data into (using the $ syntax).  By using the _PARTITIONTIME pseudo-column, users were more easily able to craft their SQL, and save costs by only addressing the necessary partition(s).  It was a major milestone for the BigQuery engineering team, and we were quick to adopt it into our data pipelines.  We rejoiced and gave each other a lot of high-fives.

Scheduling BigQuery jobs: this time using Cloud Storage & Cloud Functions

Intro

Post update: My good friend Lak over at Google has come up with a fifth option! He suggests using Cloud Dataprep to achieve the same. You can read his blog post about that over here. I had thought about using Dataprep, but because it actually spins up a Dataflow job under-the-hood, I decided to omit it from my list. That’s because it will take a lot longer to run (the cluster needs to spin up and it issues export and import commands to BigQuery), rather than issuing a query job directly to the BigQuery API. Also, there are extra costs involved with this approach (the query itself, the Dataflow job, and a Dataprep surcharge – ouch!). But, as Lak pointed out, this would be a good solution if you want to transform your data, instead of issuing a pure SQL request. However, I’d argue that can be done directly in SQL too 😉

Not so long ago, I wrote a blog post about how you can use Google Apps Script to schedule BigQuery jobs. You can find that post right here. Go have a read of it now. I promise you’ll enjoy it. The post got quite a bit of attention, and I was actually surprised that people actually take the time out to read my drivel.

It’s clear that BigQuery’s popularity is growing fast. I’m seeing more content popping up in my feeds than ever before (mostly from me because that’s all I really blog about). However, as awesome as BigQuery is, one glaring gap in its arsenal of weapons is the lack of a built-in job scheduler, or an easy way to do it outside of BigQuery.

That said however, I’m pretty sure that the boffins over in Googley-woogley-world are currently working on remedying that – by either adding schedulers to Cloud Functions, or by baking something directly into the BigQuery API itself. Or maybe both? Who knows!

Putting together the pieces: Server-side rendering with React Router v4 and Redux

Server-side rendering a React app is a miracle on-par with childbirth and modern air travel.

OK, that opening sentence was a little over-the-top. I apologise to birth mothers and those in the aviation industry.

Let me start again: server-side rendering a React app is…kind of cool.

That said, it can be a little tricky to get started, especially if you’re trying to do it with an existing app.

In this post I’ll explain one way you can implement server-side rendering (SSR) for an app that’s using  React Router v4 and Redux Thunks.

Along the way we’ll discuss the fundamental difference between JavaScript clients and servers, how it forces us to change the way we do routing, and the small “missing-link” that enables us to bridge React Router v4 with Redux thunks.

We’ll build up a simple example to demonstrate. I’m going to assume you’ve got some knowledge of:

  • React
  • Redux
  • React Router v4

However, you are not required to have knowledge of:

  • Childbirth
  • Aeronautics

Let’s do this.

Scheduling BigQuery jobs using Google Apps Script

Do you recoil in horror at the thought of running yet another mundane SQL script just so a table is automatically rebuilt for you each day in BigQuery? Can you barely remember your name first thing in the morning, let alone remember to click “Run Query” so that your boss gets the latest data refreshed in his fancy Data Studio charts, and then takes all the credit for your hard work?

Well, fear not my fellow BigQuery’ians. There’s a solution to this madness.

It’s simple.

It’s quick.

Yes, it’s Google Apps Script to the rescue.

Disclaimer: all credit for this goes to the one and only Felipe Hoffa. He ‘da man!

Extending React’s container/presentation pattern to share business logic between apps

One of the hottest JavaScript libraries out there right now is React. Whilst there are many good reasons for React being so popular, the main reason my team picked React for our new project is because of the commonality between React and React Native. The concepts and the language are the same, so if you can write React, then you can write React Native. There is also the benefit of being able put both in the same project and then share business logic between them, which is precisely what we need to be able to do.

There are a number of ambitious projects out there that aim to eliminate duplication between platforms by using a common set of React primitives. react-native-web, ReactXP and react-primitives spring to mind. However, these rely on not only a common set of components, but also a common approach to things like styling and animations.

Whilst we wait for the dust to settle on which is the best generic solution to use, my team has come up a simple approach that extends upon a pattern that is already quite well known in the React community. In this post I’ll outline this approach.

Ampersandjs at the beach

Ampersand.js

Ampersand.js was created by the good people at &yet. It is based on Backbone.js, and refers to itself as a ‘non-frameworky framework’. In a world where Javascript frontend frameworks abound, it can be difficult to choose ‘the best’ one for your project. Most of the time you would go with that which was used before, which is the most popular, or which is most familiar. In 2015 I found myself in a position where a series of small, discrete, single-purpose, front-end ‘widgets’ were proposed. The small development team was able to choose what we wanted, and Ampersand.js was selected. In this post I’ll talk about why I like using it for our projects, and what I think makes a good framework in general.

Triggering Dataflow Pipelines With Cloud Functions

Do you have an unreasonable fear of cronjobs? Find spinning up VMs to be a colossal waste of your towering intellect? Does the thought of checking a folder regularly for updates fill you with an apoplectic rage? If so, you should probably get some help. Maybe find another line of work.

In the meantime, here’s one way to ease your regular file processing anxieties. With just one application of Google Cloud Functions, eased gently up your Dataflow Pipeline, you can find lasting relief from troublesome cronjobs.