Protect Sensitive Data based on Role Based Access Control in BigQuery

Leverage policy tags and row/column level security in BigQuery datasets to secure sensitive data. With role based access controls, BigQuery users will only be able to see and access data (specific rows / columns) that they are privileged with. This works seamlessly with Google Cloud Data Catalog service.

To know more visit my blog here:

Developing a ML model with Google BigQuery to predict solar coverage

 Google BigQuery lets users develop and operationalize advanced ML models with just SQL skills. You don't have to be a Tensorflow or Keras expert or a data scientist to train ML models. Follow along with this article if you are interested to see how to develop a linear regression ML model in BigQuery to predict the solar coverage percentage for US states.

https://medium.com/@agsathya/develop-and-operationalize-ml-models-using-plain-sql-on-google-bigquery-462bc2bbe441

Achieve higher levels of cloud maturity with socially responsible IT

Photo by Joey Kyber from Pexels

What is the highest level of cloud maturity in an organization? In my opinion, it is delivering capabilities that are environmentally & socially responsible.

As a visionary, we at Google can help you develop applications and services that are not just cool but also economically and socially responsible. Google has been carbon neutral since 2007 and has committed to operating on 100% carbon-free energy (CFE) 24x7 by 2030. Google data centers are architected to use much less energy than a typical data center - read more about it here.

If you missed the Next ‘21 session on ways to reduce your carbon footprint, you can still catch up on it here.

Just by running your workloads on Google Cloud, you can have a positive impact on the planet. However, it doesn’t have to stop there. Google Cloud offers several metrics, tools and dashboards to help you understand and reduce your carbon footprint including the ability to adhere to GHG Scope 3 standard reporting requirements.

Here are 3 strategies that you can employ to reduce your carbon footprint;

1. Choose the most eco-friendly cloud region

Google publishes carbon data for all its cloud regions and you can see the CFE% and the local electricity grid’s carbon intensity. This will help you pick cleaner cloud regions (if you can) to run your workloads.

• CFE % - This metric is updated hourly and it represents the average percentage of carbon-free energy consumed in that particular region. In simple terms, you can think of this as the percentage of time your application would run on carbon-free energy. Higher the % the better it is.

• Grid Carbon Intensity - This metric indicates the gross carbon emissions from the grid per unit of energy. In layman terms, the lower this value, the better.

On the Google Cloud console, some cloud regions may have the Low CO2 marker against them. This means that the region has a Google CFE% of at least 75% and/or a grid carbon intensity of 200 gCO2eq/kWh or less.

Using a combination of the CFE% and the grid carbon intensity indicators, you now have the choice to run your workloads that are eco-friendly.

2. Leverage fully managed services where possible

Fully managed offerings inherently are typically more efficient than manually operated ones. For instance, consider the following scenarios;

• Running an application on Google Compute Engine (GCE) with auto scaling capabilities is more efficient than running it on a static size server farm on-premise

• An application that can be containerized (GKE) can offer higher server density resulting in reduced resource usage

• On the far end of the spectrum, if your workloads can leverage fully managed serverless offerings such as Cloud Run (for containerized workloads) / Cloud Functions (based on FaaS) / BigQuery (serverless data warehouse), these can provide greater efficiencies not just with economies of scale but also can reduce idle resources through smart resource allocation and scaling based on demand.

This may tie directly to your organization’s cloud maturity level and adoption. In a hybrid environment you may still have some monoliths that warrant the use of larger machines/VMs but still having this understanding will help you with your longer term “socially responsible” goals.

3. Optimize resource needs and utilization

This may sound like a cliche. 

But, just having the ability to track, monitor and report on CFE metrics is a great starting point. Solutions like Active Assist can offer insights and recommendations to optimize your cloud usage. Here are some things to ponder upon;

• Carrying on-premise mindset to cloud - Over-provisioning of VMs / server resources result not just in increased costs, but worse, negative CFE scores

• Forgot to close the faucet - Unused, running VMs are just bad practice. Having a governance model in place would go a long way in reducing emissions. For VMs that are only needed at certain times try leveraging Scheduled VMs option. The Idle VM recommendation option can be handy to identify and turn down idle VM resources.

• Try refactoring your monolith applications into microservices. Microservices are nimble, scale independently and can offer higher efficiencies and lower TCO - in stark contrast to monoliths with a significantly larger carbon footprint

• Lifting & Shifting workloads? This is not a bad option especially if you are just embarking on your cloud journey. Oftentimes, this may be the only option. However, think of creative ways to optimize it for the cloud. Can you separate the analytics capabilities? Containerize peripheral applications / services?

• Leverage cloud solutions wherever possible - Anthos for VMs, Migrate for Anthos and GKE can help you jumpstart your cloud journey and into higher levels of cloud maturity

• Consider running your standalone batch workloads in a region with higher CFE%

• Preemptible VMs can be a great way to save on operational costs plus reduce your carbon footprint - if you have long running yet stateless workloads that can take advantage of.

Ready to embark on your socially responsible app development?

What is your level of cloud maturity?