_YYYYMMDD
suffix pattern just like everyone else. This works fine, but it's quite cumbersome, has some hard limits, and your SQL can quickly becomes unruly.
Then about a year ago, the BigQuery team released ingestion time partitioning. This allowed users to partition tables based on the load/arrival time of the data, or by explicitly stating the partition to load the data into (using the $
syntax). By using the _PARTITIONTIME
pseudo-column, users were more easily able to craft their SQL, and save costs by only addressing the necessary partition(s). It was a major milestone for the BigQuery engineering team, and we were quick to adopt it into our data pipelines. We rejoiced and gave each other a lot of high-fives.
Shine's good friend Felipe Hoffa from Google was in Melbourne recently, and he took the time to catch up with our resident Google Developer Expert, Graham Polley. But, instead of just sitting down over a boring old coffee, they decided to take an iconic tram ride...
“Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis mollis auctor libero quis cursus. Mauris ultricies lacus est. Praesent interdum mauris sed turpis imperdiet, et laoreet dolor tincidunt. Donec porttitor neque a lacus condimentum ullamcorper.
Aliquam aliquet mauris a ante mattis luctus. Cras tellus ante, placerat sed ex vel, tincidunt lobortis velit.”