Compute Engine VMs

What is Compute Engine?

Compute Engine provides virtual machines running on Google's infrastructure. Unlike Cloud Run (serverless), you get full control over the operating system.

When to Use Compute Engine

Cloud Run vs Compute Engine

Step 1: Enable Compute Engine API

bash
Copy
gcloud services enable compute.googleapis.com

Step 2: Choose Your Machine Type

Machine Type Families

Common Machine Types

*Approximate us-central1 pricing

List Available Machine Types

bash
Copy
# List machine types in a zone
gcloud compute machine-types list --zones=us-central1-a

# Filter by machine family
gcloud compute machine-types list --zones=us-central1-a \
  --filter="name~^e2"

Step 3: Create a Virtual Machine

Basic VM Creation

bash
Copy
gcloud compute instances create my-vm \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --boot-disk-size=20GB

VM with Startup Script

bash
Copy
gcloud compute instances create my-web-server \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --boot-disk-size=20GB \
  --tags=http-server,https-server \
  --metadata-from-file=startup-script=startup.sh

Example Startup Script

Create startup.sh:

bash
Copy
#!/bin/bash

# Update system
apt-get update
apt-get upgrade -y

# Install Node.js
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs

# Install nginx
apt-get install -y nginx

# Clone your application
cd /var/www
git clone https://github.com/your-user/your-app.git
cd your-app

# Install dependencies and build
npm ci
npm run build

# Set up systemd service
cat > /etc/systemd/system/myapp.service << 'EOF'
[Unit]
Description=My Node.js App
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/your-app
ExecStart=/usr/bin/node server.js
Restart=on-failure
Environment=NODE_ENV=production
Environment=PORT=3000

[Install]
WantedBy=multi-user.target
EOF

# Start the service
systemctl daemon-reload
systemctl enable myapp
systemctl start myapp

# Configure nginx as reverse proxy
cat > /etc/nginx/sites-available/myapp << 'EOF'
server {
    listen 80;
    server_name _;

    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }
}
EOF

ln -sf /etc/nginx/sites-available/myapp /etc/nginx/sites-enabled/
rm -f /etc/nginx/sites-enabled/default
systemctl restart nginx

Available Images

bash
Copy
# List Debian images
gcloud compute images list --filter="family~debian"

# List Ubuntu images
gcloud compute images list --filter="family~ubuntu"

# List all public images
gcloud compute images list

# Common image families:
# - debian-12 (Debian 12 Bookworm)
# - ubuntu-2204-lts (Ubuntu 22.04)
# - ubuntu-2404-lts-amd64 (Ubuntu 24.04)
# - rocky-linux-9 (Rocky Linux 9)
# - centos-stream-9 (CentOS Stream 9)

Step 4: Configure Firewall Rules

Allow HTTP Traffic

bash
Copy
# Create firewall rule for HTTP
gcloud compute firewall-rules create allow-http \
  --direction=INGRESS \
  --priority=1000 \
  --network=default \
  --action=ALLOW \
  --rules=tcp:80 \
  --source-ranges=0.0.0.0/0 \
  --target-tags=http-server

# Create firewall rule for HTTPS
gcloud compute firewall-rules create allow-https \
  --direction=INGRESS \
  --priority=1000 \
  --network=default \
  --action=ALLOW \
  --rules=tcp:443 \
  --source-ranges=0.0.0.0/0 \
  --target-tags=https-server

Allow Custom Port

bash
Copy
# Allow port 3000
gcloud compute firewall-rules create allow-port-3000 \
  --direction=INGRESS \
  --priority=1000 \
  --network=default \
  --action=ALLOW \
  --rules=tcp:3000 \
  --source-ranges=0.0.0.0/0 \
  --target-tags=app-server

List Firewall Rules

bash
Copy
gcloud compute firewall-rules list

# Describe a specific rule
gcloud compute firewall-rules describe allow-http

Step 5: Connect to Your VM

SSH via gcloud

bash
Copy
# Connect to VM
gcloud compute ssh my-vm --zone=us-central1-a

# Connect as specific user
gcloud compute ssh user@my-vm --zone=us-central1-a

# Run a single command
gcloud compute ssh my-vm --zone=us-central1-a --command="uptime"

SSH via External Client

bash
Copy
# Get external IP
gcloud compute instances describe my-vm \
  --zone=us-central1-a \
  --format='get(networkInterfaces[0].accessConfigs[0].natIP)'

# Add SSH key to project metadata (once)
gcloud compute project-info add-metadata \
  --metadata-from-file=ssh-keys=~/.ssh/id_rsa.pub

# Or use OS Login (recommended)
gcloud compute project-info add-metadata \
  --metadata=enable-oslogin=TRUE

# Then connect normally
ssh username@EXTERNAL_IP

Transfer Files

bash
Copy
# Copy file to VM
gcloud compute scp local-file.txt my-vm:~/remote-file.txt \
  --zone=us-central1-a

# Copy file from VM
gcloud compute scp my-vm:~/remote-file.txt local-file.txt \
  --zone=us-central1-a

# Copy directory
gcloud compute scp --recurse ./my-folder my-vm:~/ \
  --zone=us-central1-a

Step 6: Manage Your VM

View VM Details

bash
Copy
# List all instances
gcloud compute instances list

# Describe specific instance
gcloud compute instances describe my-vm --zone=us-central1-a

# Get external IP
gcloud compute instances describe my-vm \
  --zone=us-central1-a \
  --format='get(networkInterfaces[0].accessConfigs[0].natIP)'

Start, Stop, and Delete

bash
Copy
# Stop VM (saves money, keeps disk)
gcloud compute instances stop my-vm --zone=us-central1-a

# Start VM
gcloud compute instances start my-vm --zone=us-central1-a

# Restart VM
gcloud compute instances reset my-vm --zone=us-central1-a

# Delete VM
gcloud compute instances delete my-vm --zone=us-central1-a

Resize VM

bash
Copy
# Stop the VM first
gcloud compute instances stop my-vm --zone=us-central1-a

# Change machine type
gcloud compute instances set-machine-type my-vm \
  --zone=us-central1-a \
  --machine-type=e2-standard-2

# Start the VM
gcloud compute instances start my-vm --zone=us-central1-a

Step 7: Static IP Addresses

Reserve Static IP

bash
Copy
# Reserve external IP
gcloud compute addresses create my-static-ip \
  --region=us-central1

# Get the IP address
gcloud compute addresses describe my-static-ip \
  --region=us-central1 \
  --format='get(address)'

Assign to VM

bash
Copy
# Create VM with static IP
gcloud compute instances create my-vm \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --address=my-static-ip

# Or assign to existing VM (requires restart)
gcloud compute instances delete-access-config my-vm \
  --zone=us-central1-a \
  --access-config-name="External NAT"

gcloud compute instances add-access-config my-vm \
  --zone=us-central1-a \
  --address=STATIC_IP_ADDRESS

Step 8: Persistent Disks

Create Additional Disk

bash
Copy
# Create disk
gcloud compute disks create my-data-disk \
  --size=100GB \
  --type=pd-ssd \
  --zone=us-central1-a

Attach to VM

bash
Copy
# Attach disk
gcloud compute instances attach-disk my-vm \
  --disk=my-data-disk \
  --zone=us-central1-a

# SSH and format the disk
gcloud compute ssh my-vm --zone=us-central1-a

# Inside the VM:
sudo mkfs.ext4 -m 0 -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/sdb
sudo mkdir -p /mnt/data
sudo mount -o discard,defaults /dev/sdb /mnt/data
sudo chmod a+w /mnt/data

# Add to fstab for persistence
echo UUID=$(sudo blkid -s UUID -o value /dev/sdb) /mnt/data ext4 discard,defaults,nofail 0 2 | sudo tee -a /etc/fstab

Disk Types

Step 9: Instance Templates and Groups

Create Instance Template

bash
Copy
gcloud compute instance-templates create my-template \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --boot-disk-size=20GB \
  --tags=http-server,https-server \
  --metadata-from-file=startup-script=startup.sh

Create Managed Instance Group

bash
Copy
# Create instance group
gcloud compute instance-groups managed create my-group \
  --base-instance-name=my-app \
  --size=2 \
  --template=my-template \
  --zone=us-central1-a

Configure Autoscaling

bash
Copy
gcloud compute instance-groups managed set-autoscaling my-group \
  --zone=us-central1-a \
  --max-num-replicas=10 \
  --min-num-replicas=2 \
  --target-cpu-utilization=0.6 \
  --cool-down-period=90

Step 10: Monitoring and Logging

View Serial Console Output

bash
Copy
# View startup logs
gcloud compute instances get-serial-port-output my-vm \
  --zone=us-central1-a

View Logs in Cloud Logging

bash
Copy
# View VM logs
gcloud logging read "resource.type=gce_instance AND resource.labels.instance_id=INSTANCE_ID" \
  --limit=50

Set Up Cloud Monitoring Agent

SSH into your VM and run:

bash
Copy
# Install Ops Agent (recommended)
curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
sudo bash add-google-cloud-ops-agent-repo.sh --also-install

This sends system metrics to Cloud Monitoring.

Common VM Setups

Node.js Application

bash
Copy
#!/bin/bash
apt-get update
apt-get install -y curl git nginx

# Install Node.js
curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs

# Clone and run app
cd /var/www
git clone https://github.com/user/app.git
cd app
npm ci --production
npm run build

# Create systemd service
cat > /etc/systemd/system/app.service << EOF
[Unit]
Description=Node.js App
After=network.target

[Service]
Type=simple
User=www-data
WorkingDirectory=/var/www/app
ExecStart=/usr/bin/node dist/server.js
Restart=on-failure
Environment=NODE_ENV=production
Environment=PORT=3000

[Install]
WantedBy=multi-user.target
EOF

systemctl enable app
systemctl start app

Docker Host

bash
Copy
#!/bin/bash
apt-get update
apt-get install -y ca-certificates curl gnupg

# Install Docker
install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg
chmod a+r /etc/apt/keyrings/docker.gpg

echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null

apt-get update
apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Start Docker
systemctl enable docker
systemctl start docker

# Pull and run your container
docker pull gcr.io/PROJECT_ID/my-app:latest
docker run -d -p 80:3000 --restart=always gcr.io/PROJECT_ID/my-app:latest

PostgreSQL Database

bash
Copy
#!/bin/bash
apt-get update
apt-get install -y postgresql postgresql-contrib

# Configure PostgreSQL
sudo -u postgres psql << EOF
CREATE USER myapp WITH PASSWORD 'secure-password';
CREATE DATABASE myappdb OWNER myapp;
GRANT ALL PRIVILEGES ON DATABASE myappdb TO myapp;
EOF

# Allow remote connections (configure firewall appropriately)
echo "listen_addresses = '*'" >> /etc/postgresql/15/main/postgresql.conf
echo "host all all 0.0.0.0/0 md5" >> /etc/postgresql/15/main/pg_hba.conf

systemctl restart postgresql

Cost Optimization

Preemptible VMs (Spot)

Up to 91% cheaper, but can be terminated at any time:

bash
Copy
gcloud compute instances create my-spot-vm \
  --zone=us-central1-a \
  --machine-type=e2-medium \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --provisioning-model=SPOT \
  --instance-termination-action=STOP

Committed Use Discounts

Commit to 1 or 3 years for 57-70% savings:

1. Go to Compute Engine → Committed use discounts
2. Purchase commitment for specific machine types
3. Discounts apply automatically

Sustained Use Discounts

Automatic discounts for VMs running 25%+ of the month - no action needed.

Right-Sizing Recommendations

bash
Copy
# View recommendations
gcloud recommender recommendations list \
  --recommender=google.compute.instance.MachineTypeRecommender \
  --location=us-central1-a \
  --project=PROJECT_ID

Troubleshooting

Can't SSH to VM

bash
Copy
# Check firewall rules
gcloud compute firewall-rules list --filter="name~ssh"

# Ensure SSH is allowed
gcloud compute firewall-rules create allow-ssh \
  --direction=INGRESS \
  --priority=1000 \
  --network=default \
  --action=ALLOW \
  --rules=tcp:22 \
  --source-ranges=0.0.0.0/0

# Check VM is running
gcloud compute instances describe my-vm --zone=us-central1-a \
  --format='get(status)'

Startup Script Not Running

bash
Copy
# Check serial output for errors
gcloud compute instances get-serial-port-output my-vm \
  --zone=us-central1-a

# View startup script logs
gcloud compute ssh my-vm --zone=us-central1-a \
  --command="sudo journalctl -u google-startup-scripts.service"

Out of Disk Space

bash
Copy
# Resize boot disk (VM must be stopped)
gcloud compute instances stop my-vm --zone=us-central1-a

gcloud compute disks resize my-vm \
  --size=50GB \
  --zone=us-central1-a

gcloud compute instances start my-vm --zone=us-central1-a

# Inside VM, extend filesystem
sudo resize2fs /dev/sda1