Gcp-Setup on Nyghtowl

How to End User OAuth for GCP

Fri, 17 Jul 2020 22:05:26 +0000

How to End User OAuth for GCP

Let’s talk about end user authentication. I’ve been digging into the authentication space a bit and have some takeaways to share.

When you access a GCP service, there is authentication to determine who you are, authorization to determine what you can do and auditing that logs what you did. IAM is where you setup roles for authorization in regards what you can do in a project.

Docker with Cloud SDK & Environment Variables

Wed, 08 Jul 2020 18:33:39 +0000

Hello again. I am sharing a couple things I learned when working with Docker recently. Main points are how to pass in environment variables, setup Google Cloud SDK in Docker and turn on debugging.

Since I haven’t written about Docker before, I’ve provided a brief overview of Docker to give context and a grounding on my pointers. I recommend doing more digging to learn more elsewhere especially considering there are many other resources out there like this Docker overview.

Setup and Invoke Cloud Functions using Python

Tue, 23 Jun 2020 17:21:13 +0000

Setup and Invoke Cloud Functions using Python

Oh the fun I’ve had learning Cloud Functions. It has actually be fun while also frustrating and definitely enlightening to explore serverless services that are supposed to make things easy. Granted I’ve been around long enough to know there is always ramp up, even when it is simple.

Cloud Functions is a service that allows you to run code on Google Cloud servers without needing to deal with server configuration or scaling. It is a pay as you go approach that means you pay for what you use and that can help optimize costs. I’ve started using it for a couple functions that I need to run multiple times and that is where it can be very valuable. There was a learning curve especially when connecting different services and this post covers key learnings on these subjects.

BigQuery Dataset Metadata Queries

Mon, 08 Jun 2020 17:49:33 +0000

BigQuery Dataset Metadata Queries

When working with tables in BigQuery, you need an understanding of a dataset structure whether it is public or you set it up and you want to review. This is a quick bit to share queries you can use to pull metadata on your datasets and tables.

In the following examples, I’m using the BigQuery public Stack Overflow database to demonstrate these commands. Change out the names as needed for the dataset and tables you are working with.

Setup and Switch Between Google Cloud Projects in the SDK

Wed, 27 May 2020 17:21:16 +0000

Setup and Switch Between Google Cloud Projects in the SDK

Providing a quick overview on how to setup and switch between Google Cloud projects with the SDK on a single machine. This is helpful when working with multiple projects (especially when collaborating) and you are using Cloud SDK.

Setup & Authenticate

The following steps are needed whether creating a project for the first time or a project already exists and you are logging into it off a computer where you will do local development. This assumes you will run all commands from a terminal.

Cloud Storage with Gsutils & Python Client Library

Sat, 29 Feb 2020 01:05:19 +0000

Cloud Storage with Gsutils & Python Client Library

BLOB (binary large object) storage is aptly named. Google Cloud Storage is a BLOB storage solution that stores unstructured data. It’s like the closet you can shove a bunch of stuff into and don’t have to organize the information before putting it inside. It’s insanely flexible in size meaning it can hold anything and doesn’t need any special processing to store it.

SSH Agent Forwarding & Remote Upstream with GitHub & GCE

Thu, 20 Feb 2020 23:13:05 +0000

SSH Agent Forwarding & Remote Upstream with GitHub & GCE

When working with a cloud-based distributed version-control system during software development like GitHub, there are a couple key configurations that will enable you to collaborate on a project and work on a remote server like GCE (Google Compute Engine). You need to provide credentials to push and pull content from GitHub which SSH agent forwarding will help you accomplish. If you are working on a forked GitHub repo (repository) then you will want to setup remote upstream to keep it up to date.

Jupyter Notebook on Compute Engine with HTTPS

Wed, 19 Feb 2020 05:51:23 +0000

Jupyter Notebook on Compute Engine with HTTPS

I wanted to run/serve Jupyter Notebook from my remote GCE instance and access it through my laptop’s browser to test and visualize Python code snippets. I’ve done this before in a previous post I wrote a few years ago but a few things have changed in the setup. For this setup, I wanted to use https to access the notebook URL which required SSL certificate setup. As stated in the title, this post steps through how I setup Jupyter Notebook to run on my GCE using HTTPS.

Setup Compute Engine with Python ML Libraries

Sat, 15 Feb 2020 01:43:42 +0000

Below I will step through how I setup a Virtual Machine on Google Compute Engine with standard requirements as well as Python and related packages. In a previous post, I went through how to setup a VM and access to it from my laptop CLI.

As mentioned, I like accessing the remote server through the CLI and below I walk through commands to setup the remote instance with basic packages for Linux instance and common Python machine learning packages. Also, the instance I worked with was based on the Debian/GNU Linux 9 image. If you use a different instance, the commands may change but the general steps are similar.

Setup Local Terminal Access to Compute Engine

Mon, 10 Feb 2020 18:53:32 +0000

Setup Local Terminal Access to Compute Engine

I’ve been playing around with the Google Cloud Platform again in a way I haven’t done for a while. It is a strange sense of deja vu and scratches that addictive learning itch. Below are the steps I took on this latest project to setup a VM (virtual machine) on GCP and login to it from my laptop terminal (CLI).

This assumes an existing GCP account. If not then sign up to follow along.