Advanced GCP Features

Cloud SQL - Managed Databases

Cloud SQL provides fully managed MySQL, PostgreSQL, and SQL Server databases.

Why Cloud SQL?

• Fully managed - Automatic backups, patches, updates
• High availability - Multi-zone failover
• Scalable - Resize without downtime
• Secure - Encryption, IAM integration
• Cloud Run integration - Direct connection via Unix socket

Create a PostgreSQL Instance

bash
Copy
# Enable Cloud SQL API
gcloud services enable sqladmin.googleapis.com

# Create instance
gcloud sql instances create my-database \
  --database-version=POSTGRES_15 \
  --tier=db-f1-micro \
  --region=us-central1 \
  --root-password=SECURE_PASSWORD \
  --storage-size=10GB \
  --storage-type=SSD

Instance Tiers

Create Database and User

bash
Copy
# Create database
gcloud sql databases create myappdb --instance=my-database

# Create user
gcloud sql users create appuser \
  --instance=my-database \
  --password=USER_PASSWORD

Connect from Cloud Run

Cloud Run connects via the Cloud SQL Auth Proxy automatically.

bash
Copy
# Deploy with Cloud SQL connection
gcloud run deploy myapp \
  --image=us-central1-docker.pkg.dev/PROJECT_ID/docker-repo/myapp:latest \
  --add-cloudsql-instances=PROJECT_ID:us-central1:my-database \
  --set-env-vars="DB_HOST=/cloudsql/PROJECT_ID:us-central1:my-database" \
  --set-env-vars="DB_USER=appuser" \
  --set-env-vars="DB_NAME=myappdb" \
  --region=us-central1

In your application:

javascript
Copy
// Node.js with pg
const { Pool } = require('pg');

const pool = new Pool({
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD, // From Secret Manager
  database: process.env.DB_NAME,
  host: process.env.DB_HOST, // Unix socket path
});

Connect from Local Development

bash
Copy
# Install Cloud SQL Proxy
curl -o cloud-sql-proxy https://storage.googleapis.com/cloud-sql-connectors/cloud-sql-proxy/v2.8.0/cloud-sql-proxy.linux.amd64
chmod +x cloud-sql-proxy

# Start proxy
./cloud-sql-proxy PROJECT_ID:us-central1:my-database

# Connect via localhost:5432
psql -h localhost -U appuser -d myappdb

High Availability

bash
Copy
# Create HA instance
gcloud sql instances create my-database \
  --database-version=POSTGRES_15 \
  --tier=db-custom-2-4096 \
  --region=us-central1 \
  --availability-type=REGIONAL \
  --root-password=SECURE_PASSWORD

Backups

bash
Copy
# Enable automated backups
gcloud sql instances patch my-database \
  --backup-start-time=02:00

# Create on-demand backup
gcloud sql backups create --instance=my-database

# List backups
gcloud sql backups list --instance=my-database

Cloud Storage - Object Storage

Store files, images, backups, and static assets.

Create a Bucket

bash
Copy
# Create bucket (name must be globally unique)
gcloud storage buckets create gs://my-unique-bucket-name \
  --location=us-central1 \
  --uniform-bucket-level-access

# With specific storage class
gcloud storage buckets create gs://my-archive-bucket \
  --location=us \
  --storage-class=NEARLINE

Storage Classes

Upload and Download Files

bash
Copy
# Upload file
gcloud storage cp local-file.txt gs://my-bucket/

# Upload directory
gcloud storage cp -r ./my-folder gs://my-bucket/

# Download file
gcloud storage cp gs://my-bucket/file.txt ./local-file.txt

# List contents
gcloud storage ls gs://my-bucket/

# Delete file
gcloud storage rm gs://my-bucket/file.txt

Make Bucket Public (Static Website)

bash
Copy
# Make all objects public
gcloud storage buckets add-iam-policy-binding gs://my-bucket \
  --member=allUsers \
  --role=roles/storage.objectViewer

# Set website configuration
gcloud storage buckets update gs://my-bucket \
  --web-main-page-suffix=index.html \
  --web-not-found-page=404.html

Signed URLs (Temporary Access)

javascript
Copy
// Node.js - Generate signed URL
const { Storage } = require('@google-cloud/storage');
const storage = new Storage();

async function generateSignedUrl(bucketName, fileName) {
  const options = {
    version: 'v4',
    action: 'read',
    expires: Date.now() + 15 * 60 * 1000, // 15 minutes
  };

  const [url] = await storage
    .bucket(bucketName)
    .file(fileName)
    .getSignedUrl(options);

  return url;
}

Upload from Application

javascript
Copy
// Node.js - Upload file
const { Storage } = require('@google-cloud/storage');
const storage = new Storage();

async function uploadFile(bucketName, filePath, destFileName) {
  await storage.bucket(bucketName).upload(filePath, {
    destination: destFileName,
    metadata: {
      contentType: 'image/jpeg',
      cacheControl: 'public, max-age=31536000',
    },
  });
}

CORS Configuration

bash
Copy
# Create cors.json
cat > cors.json << 'EOF'
[
  {
    "origin": ["https://example.com"],
    "method": ["GET", "POST", "PUT"],
    "responseHeader": ["Content-Type"],
    "maxAgeSeconds": 3600
  }
]
EOF

# Apply CORS config
gcloud storage buckets update gs://my-bucket --cors-file=cors.json

Secret Manager

Securely store API keys, passwords, and other secrets.

Create and Access Secrets

bash
Copy
# Enable Secret Manager API
gcloud services enable secretmanager.googleapis.com

# Create secret from value
echo -n "my-api-key-value" | \
  gcloud secrets create API_KEY --data-file=-

# Create secret from file
gcloud secrets create DB_PASSWORD --data-file=./password.txt

# Access secret value
gcloud secrets versions access latest --secret=API_KEY

Secret Versions

bash
Copy
# Add new version
echo -n "new-api-key-value" | \
  gcloud secrets versions add API_KEY --data-file=-

# List versions
gcloud secrets versions list API_KEY

# Access specific version
gcloud secrets versions access 1 --secret=API_KEY

# Disable a version
gcloud secrets versions disable 1 --secret=API_KEY

Use in Cloud Run

bash
Copy
# Deploy with secrets
gcloud run deploy myapp \
  --image=us-central1-docker.pkg.dev/PROJECT_ID/docker-repo/myapp:latest \
  --set-secrets="API_KEY=API_KEY:latest,DB_PASSWORD=DB_PASSWORD:latest" \
  --region=us-central1

Secrets are available as environment variables in your application.

Grant Access

bash
Copy
# Get Cloud Run service account
SERVICE_ACCOUNT=$(gcloud run services describe myapp \
  --region=us-central1 \
  --format='value(spec.template.spec.serviceAccountName)')

# Grant access to secret
gcloud secrets add-iam-policy-binding API_KEY \
  --member="serviceAccount:$SERVICE_ACCOUNT" \
  --role="roles/secretmanager.secretAccessor"

Cloud Monitoring

Monitor your applications and infrastructure.

Key Metrics

Cloud Run automatically reports:

• Request count
• Request latency
• Instance count
• Memory utilization
• CPU utilization
• Container startup latency

View Metrics

1. Go to Monitoring → Dashboards
2. Select Cloud Run or Compute Engine
3. View pre-built dashboards

Create Custom Dashboard

bash
Copy
# Via Console: Monitoring → Dashboards → Create Dashboard
# Add widgets for the metrics you care about

Alerting Policies

bash
Copy
# Create alert for high error rate
gcloud alpha monitoring policies create \
  --display-name="High Error Rate" \
  --condition-display-name="Error rate > 1%" \
  --condition-filter='resource.type="cloud_run_revision" AND metric.type="run.googleapis.com/request_count" AND metric.labels.response_code_class="5xx"' \
  --condition-threshold-value=0.01 \
  --condition-threshold-comparison=COMPARISON_GT \
  --notification-channels=CHANNEL_ID

Create Notification Channel

1. Go to Monitoring → Alerting → Edit notification channels
2. Add email, Slack, PagerDuty, etc.

Cloud Logging

Centralized logging for all GCP resources.

View Logs

bash
Copy
# Cloud Run logs
gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=myapp" \
  --limit=100 \
  --format="table(timestamp, textPayload)"

# Filter by severity
gcloud logging read "resource.type=cloud_run_revision AND severity>=ERROR" \
  --limit=50

Log-Based Metrics

Create metrics from log entries:

1. Go to Logging → Logs Explorer
2. Create a query
3. Click Create Metric
4. Use in dashboards and alerts

Export Logs

bash
Copy
# Export to Cloud Storage
gcloud logging sinks create my-storage-sink \
  storage.googleapis.com/my-logs-bucket \
  --log-filter="resource.type=cloud_run_revision"

# Export to BigQuery
gcloud logging sinks create my-bq-sink \
  bigquery.googleapis.com/projects/PROJECT_ID/datasets/logs_dataset \
  --log-filter="resource.type=cloud_run_revision"

VPC and Private Networking

Serverless VPC Access

Connect Cloud Run to private resources:

bash
Copy
# Create VPC connector
gcloud compute networks vpc-access connectors create my-connector \
  --region=us-central1 \
  --network=default \
  --range=10.8.0.0/28

# Deploy with VPC access
gcloud run deploy myapp \
  --image=us-central1-docker.pkg.dev/PROJECT_ID/docker-repo/myapp:latest \
  --vpc-connector=my-connector \
  --vpc-egress=private-ranges-only \
  --region=us-central1

Private Cloud SQL

bash
Copy
# Enable private IP for Cloud SQL
gcloud sql instances patch my-database \
  --network=default \
  --no-assign-ip

# Connect using private IP from Cloud Run
gcloud run deploy myapp \
  --add-cloudsql-instances=PROJECT_ID:us-central1:my-database \
  --vpc-connector=my-connector \
  --set-env-vars="DB_HOST=PRIVATE_IP"

Identity-Aware Proxy (IAP)

Protect applications with Google authentication.

Enable IAP for Cloud Run

bash
Copy
# Create OAuth consent screen first (via Console)
# APIs & Services → OAuth consent screen

# Enable IAP
gcloud run services update myapp \
  --region=us-central1 \
  --no-allow-unauthenticated

# Grant access to specific users
gcloud run services add-iam-policy-binding myapp \
  --region=us-central1 \
  --member="user:[email protected]" \
  --role="roles/run.invoker"

Error Reporting

Automatic error detection and grouping.

Enable Error Reporting

bash
Copy
gcloud services enable clouderrorreporting.googleapis.com

Errors are automatically detected from logs with stack traces.

View Errors

1. Go to Error Reporting
2. See grouped errors with:
   • Stack traces
   • Affected users
   • First/last occurrence
   • Error count

Custom Error Reporting (Node.js)

javascript
Copy
const { ErrorReporting } = require('@google-cloud/error-reporting');
const errors = new ErrorReporting();

app.use((err, req, res, next) => {
  errors.report(err);
  res.status(500).send('Something broke!');
});

Cost Management

Budget Alerts

bash
Copy
# Create budget (via Console is easier)
# Billing → Budgets & alerts → Create budget

Cost Breakdown

View costs by service:

1. Billing → Reports
2. Filter by project, service, SKU

Cost Optimization Tips

1. Use committed use discounts - 1-3 year commitments
2. Right-size resources - Don't over-provision
3. Delete unused resources - IPs, disks, snapshots
4. Use Cloud Run - Pay only for actual requests
5. Set min instances to 0 - When cold starts are acceptable
6. Use regional resources - Avoid multi-region unless needed

Resource Cleanup

bash
Copy
# Find unused resources
gcloud compute addresses list --filter="status=RESERVED"
gcloud compute disks list --filter="-users:*"

# Delete unused
gcloud compute addresses delete UNUSED_IP --region=us-central1
gcloud compute disks delete UNUSED_DISK --zone=us-central1-a

Production Checklist

Security

• Use Secret Manager for sensitive values
• Enable Cloud SQL IAM authentication
• Set up VPC for private resources
• Enable Cloud Armor for DDoS protection
• Review IAM permissions (least privilege)
• Enable audit logging

Reliability

• Enable Cloud SQL high availability
• Set up Cloud Run min instances (avoid cold starts)
• Configure health checks
• Set up alerting policies
• Create backup strategy

Performance

• Enable Cloud CDN for static assets
• Use connection pooling for databases
• Set appropriate memory/CPU for Cloud Run
• Monitor and optimize slow queries

Monitoring

• Create monitoring dashboard
• Set up error alerting
• Configure log-based metrics
• Enable uptime checks
• Set up budget alerts

Deployment

• Automate with Cloud Build
• Use traffic splitting for rollouts
• Set up staging environment
• Configure branch protection

Complete Production Setup Script

bash
Copy
#!/bin/bash
PROJECT_ID="my-project"
REGION="us-central1"
SERVICE_NAME="myapp"

# Enable APIs
gcloud services enable \
  run.googleapis.com \
  sqladmin.googleapis.com \
  secretmanager.googleapis.com \
  cloudbuild.googleapis.com \
  monitoring.googleapis.com

# Create Cloud SQL instance
gcloud sql instances create ${SERVICE_NAME}-db \
  --database-version=POSTGRES_15 \
  --tier=db-custom-2-4096 \
  --region=$REGION \
  --availability-type=REGIONAL \
  --storage-size=20GB

# Create database and user
gcloud sql databases create ${SERVICE_NAME}db --instance=${SERVICE_NAME}-db
DB_PASSWORD=$(openssl rand -base64 32)
gcloud sql users create appuser \
  --instance=${SERVICE_NAME}-db \
  --password=$DB_PASSWORD

# Store password in Secret Manager
echo -n "$DB_PASSWORD" | gcloud secrets create DB_PASSWORD --data-file=-

# Create storage bucket
gcloud storage buckets create gs://${PROJECT_ID}-${SERVICE_NAME}-uploads \
  --location=$REGION \
  --uniform-bucket-level-access

# Create VPC connector
gcloud compute networks vpc-access connectors create ${SERVICE_NAME}-connector \
  --region=$REGION \
  --network=default \
  --range=10.8.0.0/28

# Deploy Cloud Run service
gcloud run deploy $SERVICE_NAME \
  --image=us-central1-docker.pkg.dev/${PROJECT_ID}/docker-repo/${SERVICE_NAME}:latest \
  --region=$REGION \
  --add-cloudsql-instances=${PROJECT_ID}:${REGION}:${SERVICE_NAME}-db \
  --vpc-connector=${SERVICE_NAME}-connector \
  --set-secrets="DB_PASSWORD=DB_PASSWORD:latest" \
  --set-env-vars="DB_HOST=/cloudsql/${PROJECT_ID}:${REGION}:${SERVICE_NAME}-db" \
  --set-env-vars="DB_USER=appuser" \
  --set-env-vars="DB_NAME=${SERVICE_NAME}db" \
  --set-env-vars="STORAGE_BUCKET=${PROJECT_ID}-${SERVICE_NAME}-uploads" \
  --min-instances=1 \
  --max-instances=10 \
  --memory=512Mi \
  --cpu=1 \
  --allow-unauthenticated

echo "Deployment complete!"
echo "Service URL: $(gcloud run services describe $SERVICE_NAME --region=$REGION --format='value(status.url)')"