GitOps Workflow with ArgoCD

Introduction

"Just SSH into the server and run kubectl apply" works until it doesn't. Manual deployments don't scale, create inconsistencies between environments, and make auditing impossible. When something breaks at 3am, you can't remember what you deployed or when.

GitOps solves this by making Git the single source of truth for your infrastructure and applications. Every change goes through Git. Every deployment is auditable. Every environment can be recreated from scratch. ArgoCD makes GitOps practical by continuously monitoring Git repositories and automatically syncing your Kubernetes clusters.

This guide walks through implementing production-grade GitOps with ArgoCD, from basic concepts to advanced patterns that scale across multiple teams and environments.

What is GitOps?

Core Principles

1. Declarative Configuration

Everything is defined as code (YAML, Helm charts, Kustomize), describing the desired state.

# What you want, not how to get there
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3  # I want 3 replicas
  template:
    spec:
      containers:
      - name: app
        image: myapp:v2.1.0

2. Version Control as Source of Truth

Git repository contains the complete state of your system. What's in Git is what's running.

Git Repository         Kubernetes Cluster
--------------         ------------------
main branch      =     Production
staging branch   =     Staging
develop branch   =     Development

3. Automated Deployment

Software agents automatically apply changes from Git to clusters. No manual kubectl apply.

4. Continuous Reconciliation

Agents continuously monitor and correct drift, ensuring actual state matches desired state.

GitOps vs Traditional CI/CD

Traditional CI/CD:

Developer → Push Code → CI Build → CI Deploy → Production
                                    ↑
                                (kubectl apply)

GitOps:

Developer → Push Code → CI Build → Update Git Manifest
                                          ↓
                                    ArgoCD watches
                                          ↓
                                    Auto-sync to Production

Key Differences:

Traditional GitOps CI/CD pushes changes CD agent pulls changes Cluster credentials in CI No cluster credentials needed in CI Hard to audit what's deployed Git history is deployment history Drift detection requires tools Built-in drift detection Rollback via CI pipeline Rollback via Git revert

ArgoCD Architecture

Core Components

┌─────────────────────────────────────────────────────────┐
│                    ArgoCD Architecture                   │
└─────────────────────────────────────────────────────────┘

┌──────────────────┐         ┌──────────────────┐
│   Git Repository │◄────────│   ArgoCD Server  │
│                  │         │                  │
│  - YAML manifests│         │  - API Server    │
│  - Helm charts   │         │  - Web UI        │
│  - Kustomize     │         │  - Webhook       │
└──────────────────┘         └────────┬─────────┘
                                      │
                                      │ Watches
                                      ↓
                    ┌─────────────────────────────┐
                    │    Application Controller   │
                    │                             │
                    │  - Monitors applications    │
                    │  - Detects drift            │
                    │  - Auto-syncs              │
                    └─────────────┬───────────────┘
                                  │
                                  │ Deploys
                                  ↓
                    ┌─────────────────────────────┐
                    │    Kubernetes Cluster       │
                    │                             │
                    │  - Namespaces               │
                    │  - Deployments              │
                    │  - Services                 │
                    └─────────────────────────────┘

Key Concepts

Application: Represents a deployed application or service

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
spec:
  source:
    repoURL: https://github.com/company/apps
    path: myapp/base
    targetRevision: HEAD
  destination:
    server: https://kubernetes.default.svc
    namespace: production

Project: Logical grouping of applications with RBAC

Sync: Process of making cluster state match Git state

Sync Status: Comparison between Git and cluster (Synced, OutOfSync)

Health Status: Whether deployed resources are healthy (Healthy, Progressing, Degraded)

Installation and Setup

Install ArgoCD on Kubernetes

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for pods to be ready
kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=argocd-server -n argocd --timeout=300s

# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Port forward to access UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Access UI at https://localhost:8080
# Username: admin
# Password: (from command above)

Install ArgoCD CLI

# macOS
brew install argocd

# Linux
curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64
chmod +x /usr/local/bin/argocd

# Login via CLI
argocd login localhost:8080 --username admin --password <password> --insecure

Production Configuration

# argocd-cm.yaml - ArgoCD configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  # Enable automatic sync for all apps
  application.instanceLabelKey: argocd.argoproj.io/instance
  
  # Timeout settings
  timeout.reconciliation: 180s
  timeout.hard.reconciliation: 0
  
  # Resource tracking method
  application.resourceTrackingMethod: annotation
  
  # Git webhook secret
  webhook.github.secret: <secret>
  
  # SSO configuration (optional)
  url: https://argocd.company.com
  dex.config: |
    connectors:
    - type: github
      id: github
      name: GitHub
      config:
        clientID: $dex.github.clientId
        clientSecret: $dex.github.clientSecret
        orgs:
        - name: mycompany

# argocd-rbac-cm.yaml - RBAC configuration
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.default: role:readonly
  policy.csv: |
    # Admin role
    p, role:admin, applications, *, */*, allow
    p, role:admin, clusters, *, *, allow
    p, role:admin, repositories, *, *, allow
    
    # Developer role - can deploy to dev/staging only
    p, role:developer, applications, get, */*, allow
    p, role:developer, applications, sync, dev/*, allow
    p, role:developer, applications, sync, staging/*, allow
    
    # Grant admin role to platform team
    g, platform-team, role:admin
    
    # Grant developer role to engineering teams
    g, backend-team, role:developer
    g, frontend-team, role:developer

Basic Application Deployment

Repository Structure

gitops-repo/
├── apps/
│   ├── web-app/
│   │   ├── base/
│   │   │   ├── deployment.yaml
│   │   │   ├── service.yaml
│   │   │   └── kustomization.yaml
│   │   └── overlays/
│   │       ├── dev/
│   │       │   └── kustomization.yaml
│   │       ├── staging/
│   │       │   └── kustomization.yaml
│   │       └── prod/
│   │           └── kustomization.yaml
│   └── api-service/
│       └── ...
└── argocd/
    ├── applications/
    │   ├── web-app-dev.yaml
    │   ├── web-app-staging.yaml
    │   └── web-app-prod.yaml
    └── projects/
        └── default-project.yaml

Simple Deployment Manifest

# apps/web-app/base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  labels:
    app: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web
        image: nginx:1.21
        ports:
        - containerPort: 80
        resources:
          requests:
            cpu: 100m
            memory: 128Mi
          limits:
            cpu: 200m
            memory: 256Mi
---
apiVersion: v1
kind: Service
metadata:
  name: web-app
spec:
  selector:
    app: web-app
  ports:
  - port: 80
    targetPort: 80
  type: ClusterIP

Kustomize Overlays

# apps/web-app/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - service.yaml

# apps/web-app/overlays/dev/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: dev
resources:
  - ../../base
patches:
  - patch: |-
      - op: replace
        path: /spec/replicas
        value: 1
    target:
      kind: Deployment
      name: web-app
commonLabels:
  environment: dev

# apps/web-app/overlays/prod/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: production
resources:
  - ../../base
patches:
  - patch: |-
      - op: replace
        path: /spec/replicas
        value: 3
    target:
      kind: Deployment
      name: web-app
  - patch: |-
      - op: add
        path: /spec/template/spec/containers/0/resources/requests/cpu
        value: 500m
      - op: add
        path: /spec/template/spec/containers/0/resources/requests/memory
        value: 512Mi
    target:
      kind: Deployment
      name: web-app
commonLabels:
  environment: production

ArgoCD Application Definition

# argocd/applications/web-app-prod.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-prod
  namespace: argocd
  # Add finalizer to ensure cascade deletion
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  # Project this application belongs to
  project: default
  
  # Source configuration
  source:
    repoURL: https://github.com/company/gitops-repo
    targetRevision: main
    path: apps/web-app/overlays/prod
  
  # Destination cluster and namespace
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  
  # Sync policy
  syncPolicy:
    automated:
      prune: true      # Delete resources not in Git
      selfHeal: true   # Revert manual changes
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true  # Auto-create namespace
      - PrunePropagationPolicy=foreground
      - PruneLast=true
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m
  
  # Ignore differences in certain fields
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers:
        - /spec/replicas  # Ignore if HPA controls replicas

Deploy Application via CLI

# Create application
argocd app create web-app-prod \
  --repo https://github.com/company/gitops-repo \
  --path apps/web-app/overlays/prod \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production \
  --sync-policy automated \
  --auto-prune \
  --self-heal

# View application status
argocd app get web-app-prod

# Manually trigger sync
argocd app sync web-app-prod

# View sync history
argocd app history web-app-prod

# Rollback to previous revision
argocd app rollback web-app-prod

Multi-Environment Strategy

Pattern 1: Separate Git Branches

Repository Structure:
├── main branch      → Production
├── staging branch   → Staging
└── develop branch   → Development

Workflow:
develop → staging → main
(auto-deploy) (manual) (manual)

# Production application points to main branch
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-prod
spec:
  source:
    repoURL: https://github.com/company/app
    targetRevision: main  # Production from main
    path: manifests

---
# Staging application points to staging branch
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-staging
spec:
  source:
    repoURL: https://github.com/company/app
    targetRevision: staging  # Staging from staging branch
    path: manifests

Pros:

Simple to understand
Natural Git workflow (branch = environment)
Easy to see what's in each environment

Cons:

Merge conflicts between branches
Configuration drift between environments
Harder to promote changes

Pattern 2: Kustomize Overlays (Recommended)

Repository Structure:
apps/
└── web-app/
    ├── base/           # Common configuration
    └── overlays/
        ├── dev/        # Dev-specific patches
        ├── staging/    # Staging-specific patches
        └── prod/       # Prod-specific patches

All from main branch

# Development
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-dev
spec:
  source:
    repoURL: https://github.com/company/gitops
    targetRevision: main
    path: apps/web-app/overlays/dev

---
# Production
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-prod
spec:
  source:
    repoURL: https://github.com/company/gitops
    targetRevision: main
    path: apps/web-app/overlays/prod

Pros:

Single source of truth (main branch)
DRY principle (base + overlays)
Easy to see differences between environments
Smooth promotion (same Git commit)

Cons:

Requires understanding Kustomize
All environments update together

Pattern 3: Helm with Values Files

Repository Structure:
charts/
└── web-app/
    ├── Chart.yaml
    ├── templates/
    └── values/
        ├── dev.yaml
        ├── staging.yaml
        └── prod.yaml

# Production with Helm values
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-prod
spec:
  source:
    repoURL: https://github.com/company/gitops
    targetRevision: main
    path: charts/web-app
    helm:
      valueFiles:
        - values/prod.yaml
      parameters:
        - name: image.tag
          value: v2.1.0

Pros:

Familiar to Helm users
Powerful templating
Can use external Helm charts

Cons:

Additional complexity
Harder to debug template issues

App of Apps Pattern

Manage multiple applications declaratively.

# argocd/app-of-apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: app-of-apps
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/company/gitops-repo
    targetRevision: main
    path: argocd/applications
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Directory structure:

argocd/
├── app-of-apps.yaml        # Root application
└── applications/
    ├── web-app-dev.yaml
    ├── web-app-staging.yaml
    ├── web-app-prod.yaml
    ├── api-service-dev.yaml
    ├── api-service-staging.yaml
    └── api-service-prod.yaml

Benefits:

Declare all applications in Git
Bootstrap entire cluster from one application
Applications managed by ArgoCD itself

ApplicationSet: Advanced Patterns

Generate multiple applications from templates.

Pattern 1: Multiple Clusters

# argocd/applicationsets/web-app-multi-cluster.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: web-app-multi-cluster
  namespace: argocd
spec:
  generators:
    - list:
        elements:
          - cluster: us-east-1
            url: https://kubernetes.us-east-1.company.com
            environment: prod
          - cluster: us-west-2
            url: https://kubernetes.us-west-2.company.com
            environment: prod
          - cluster: eu-west-1
            url: https://kubernetes.eu-west-1.company.com
            environment: prod
  
  template:
    metadata:
      name: 'web-app-{{cluster}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/company/gitops
        targetRevision: main
        path: apps/web-app/overlays/{{environment}}
      destination:
        server: '{{url}}'
        namespace: production
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

Pattern 2: Monorepo with Multiple Apps

# argocd/applicationsets/all-apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: all-production-apps
  namespace: argocd
spec:
  generators:
    - git:
        repoURL: https://github.com/company/gitops
        revision: main
        directories:
          - path: apps/*/overlays/prod
  
  template:
    metadata:
      name: '{{path.basename}}-prod'
    spec:
      project: default
      source:
        repoURL: https://github.com/company/gitops
        targetRevision: main
        path: '{{path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: production
      syncPolicy:
        automated:
          prune: true
          selfHeal: true

This automatically creates applications for:

apps/web-app/overlays/prod → web-app-prod
apps/api-service/overlays/prod → api-service-prod
apps/worker/overlays/prod → worker-prod

Pattern 3: Pull Request Environments

# argocd/applicationsets/pr-environments.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: pr-environments
  namespace: argocd
spec:
  generators:
    - pullRequest:
        github:
          owner: company
          repo: app
          tokenRef:
            secretName: github-token
            key: token
          labels:
            - preview  # Only PRs with "preview" label
  
  template:
    metadata:
      name: 'pr-{{number}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/company/app
        targetRevision: '{{head_sha}}'
        path: manifests
      destination:
        server: https://kubernetes.default.svc
        namespace: 'pr-{{number}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

CI/CD Integration

GitHub Actions Integration

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches:
      - main
    paths:
      - 'src/**'

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Build and push Docker image
        run: |
          docker build -t myapp:${{ github.sha }} .
          docker push myapp:${{ github.sha }}
      
      - name: Update manifest
        run: |
          # Clone GitOps repo
          git clone https://github.com/company/gitops-repo
          cd gitops-repo
          
          # Update image tag
          cd apps/web-app/base
          kustomize edit set image myapp=myapp:${{ github.sha }}
          
          # Commit and push
          git config user.name "GitHub Actions"
          git config user.email "[email protected]"
          git add .
          git commit -m "Update web-app image to ${{ github.sha }}"
          git push
      
      - name: Trigger ArgoCD sync (optional)
        run: |
          argocd app sync web-app-prod --auth-token ${{ secrets.ARGOCD_TOKEN }}

GitLab CI Integration

# .gitlab-ci.yml
stages:
  - build
  - deploy

build:
  stage: build
  script:
    - docker build -t myapp:$CI_COMMIT_SHA .
    - docker push myapp:$CI_COMMIT_SHA

update-manifest:
  stage: deploy
  script:
    - git clone https://gitlab.com/company/gitops-repo
    - cd gitops-repo
    - yq -i '.spec.source.helm.parameters[] |= select(.name == "image.tag").value = env(CI_COMMIT_SHA)' argocd/applications/web-app-prod.yaml
    - git add .
    - git commit -m "Update image to $CI_COMMIT_SHA"
    - git push https://[email protected]/company/gitops-repo
  only:
    - main

Image Updater

Automatically update image tags in Git:

# argocd-image-updater-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-image-updater-config
  namespace: argocd
data:
  registries.conf: |
    registries:
    - name: Docker Hub
      prefix: docker.io
      api_url: https://registry-1.docker.io
      ping: yes
      credentials: secret:argocd/dockerhub-creds

# Application with image updater annotations
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app-prod
  annotations:
    argocd-image-updater.argoproj.io/image-list: myapp=myapp
    argocd-image-updater.argoproj.io/myapp.update-strategy: semver
    argocd-image-updater.argoproj.io/myapp.image-name: docker.io/company/myapp
    argocd-image-updater.argoproj.io/myapp.tag-pattern: ^v(\d+\.\d+\.\d+)$
    argocd-image-updater.argoproj.io/write-back-method: git
spec:
  # ... rest of spec

Advanced Features

Progressive Delivery with Rollouts

# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: web-app
spec:
  replicas: 5
  strategy:
    canary:
      steps:
        - setWeight: 20
        - pause: {duration: 1m}
        - setWeight: 40
        - pause: {duration: 1m}
        - setWeight: 60
        - pause: {duration: 1m}
        - setWeight: 80
        - pause: {duration: 1m}
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web
        image: myapp:v2.0.0

Blue-Green Deployment

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: web-app
spec:
  replicas: 3
  strategy:
    blueGreen:
      activeService: web-app-active
      previewService: web-app-preview
      autoPromotionEnabled: false
      scaleDownDelaySeconds: 30
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: web
        image: myapp:v2.0.0

Resource Hooks

Execute actions at specific sync phases:

# pre-sync-job.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: database-migration
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  template:
    spec:
      containers:
      - name: migrate
        image: myapp:v2.0.0
        command: ["python", "manage.py", "migrate"]
      restartPolicy: Never

Hook phases:

PreSync: Before manifest application
Sync: During manifest application
Skip: Skip resource from sync
PostSync: After all resources are healthy
SyncFail: On sync failure

Sync Waves

Control resource creation order:

# 1. Create namespace first
apiVersion: v1
kind: Namespace
metadata:
  name: production
  annotations:
    argocd.argoproj.io/sync-wave: "0"

---
# 2. Create ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  annotations:
    argocd.argoproj.io/sync-wave: "1"
data:
  config.json: |
    {"env": "prod"}

---
# 3. Create Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  annotations:
    argocd.argoproj.io/sync-wave: "2"
spec:
  # ... deployment spec

Resources are created in wave order: 0, then 1, then 2.

Security Best Practices

Repository Credentials

# Store credentials in Secret
apiVersion: v1
kind: Secret
metadata:
  name: repo-credentials
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: git
  url: https://github.com/company/gitops-repo
  password: <github-token>
  username: not-used

Sealed Secrets

Encrypt secrets before committing to Git:

# Install Sealed Secrets controller
kubectl apply -f https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.18.0/controller.yaml

# Create sealed secret
kubectl create secret generic db-credentials \
  --from-literal=username=admin \
  --from-literal=password=secret \
  --dry-run=client -o yaml | \
  kubeseal -o yaml > sealed-secret.yaml

# Commit sealed-secret.yaml to Git
git add sealed-secret.yaml
git commit -m "Add database credentials"

# sealed-secret.yaml (safe to commit)
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: db-credentials
  namespace: production
spec:
  encryptedData:
    username: AgBh4R5... (encrypted)
    password: AgCk2D3... (encrypted)

RBAC Configuration

# Restrict team access to their namespaces
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-rbac-cm
  namespace: argocd
data:
  policy.csv: |
    # Backend team can only manage backend namespace
    p, role:backend-team, applications, *, backend/*, allow
    p, role:backend-team, applications, get, */*, allow
    g, backend-team, role:backend-team
    
    # Frontend team can only manage frontend namespace
    p, role:frontend-team, applications, *, frontend/*, allow
    p, role:frontend-team, applications, get, */*, allow
    g, frontend-team, role:frontend-team

Project Restrictions

# Restrict deployment destinations
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: backend-project
  namespace: argocd
spec:
  description: Backend team applications
  
  # Allowed source repositories
  sourceRepos:
    - 'https://github.com/company/backend-*'
  
  # Allowed destination clusters/namespaces
  destinations:
    - namespace: 'backend-*'
      server: https://kubernetes.default.svc
  
  # Allowed resource types
  clusterResourceWhitelist:
    - group: ''
      kind: Namespace
  
  namespaceResourceWhitelist:
    - group: 'apps'
      kind: Deployment
    - group: ''
      kind: Service
    - group: ''
      kind: ConfigMap
  
  # Deny certain resources
  namespaceResourceBlacklist:
    - group: ''
      kind: ResourceQuota

Monitoring and Troubleshooting

Prometheus Metrics

# ServiceMonitor for ArgoCD metrics
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: argocd-metrics
  namespace: argocd
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: argocd-server
  endpoints:
    - port: metrics
      interval: 30s

Key metrics to monitor:

argocd_app_info: Application metadata
argocd_app_sync_total: Sync count by status
argocd_app_reconcile_count: Reconciliation attempts
argocd_app_health_status: Application health

Grafana Dashboard

{
  "dashboard": {
    "title": "ArgoCD Overview",
    "panels": [
      {
        "title": "Application Health",
        "targets": [{
          "expr": "count(argocd_app_info{health_status='Healthy'})"
        }]
      },
      {
        "title": "Sync Status",
        "targets": [{
          "expr": "count by (sync_status) (argocd_app_info)"
        }]
      },
      {
        "title": "Sync History",
        "targets": [{
          "expr": "rate(argocd_app_sync_total[5m])"
        }]
      }
    ]
  }
}

Common Issues

Issue 1: Application Stuck in Progressing

# Check application events
argocd app get myapp --show-operation

# Check pod logs
kubectl logs -n production deployment/myapp

# Check events
kubectl get events -n production --sort-by='.lastTimestamp'

Issue 2: OutOfSync but Auto-Sync Not Working

# Check sync policy
argocd app get myapp -o yaml | grep syncPolicy -A 10

# Manually trigger sync
argocd app sync myapp

# Check application controller logs
kubectl logs -n argocd deployment/argocd-application-controller

Issue 3: Failed to Load Live State

# Check RBAC permissions
kubectl auth can-i list deployments -n production --as system:serviceaccount:argocd:argocd-application-controller

# Check ArgoCD service account
kubectl describe serviceaccount argocd-application-controller -n argocd

Debugging Commands

# View application details
argocd app get myapp

# View application manifest
argocd app manifests myapp

# Compare live vs desired state
argocd app diff myapp

# View sync history
argocd app history myapp

# View application logs
argocd app logs myapp

# Suspend auto-sync temporarily
argocd app set myapp --sync-policy none

# Re-enable auto-sync
argocd app set myapp --sync-policy automated

Best Practices

1. Repository Structure

Use monorepo for simplicity:

gitops-repo/
├── apps/                    # Application manifests
│   ├── web-app/
│   ├── api-service/
│   └── worker/
├── infrastructure/          # Infrastructure components
│   ├── ingress-nginx/
│   ├── cert-manager/
│   └── prometheus/
└── argocd/                  # ArgoCD configuration
    ├── applications/
    ├── applicationsets/
    └── projects/

2. Sync Policies

Production:

syncPolicy:
  automated:
    prune: false      # Manual approval for deletions
    selfHeal: false   # Allow hotfixes temporarily
  syncOptions:
    - CreateNamespace=true

Development:

syncPolicy:
  automated:
    prune: true       # Auto-cleanup
    selfHeal: true    # Always match Git
  syncOptions:
    - CreateNamespace=true

3. Health Checks

# Custom health check for CRDs
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  resource.customizations: |
    mycompany.io/MyCustomResource:
      health.lua: |
        hs = {}
        if obj.status ~= nil and obj.status.phase ~= nil then
          if obj.status.phase == "Running" then
            hs.status = "Healthy"
            hs.message = "Resource is running"
            return hs
          end
        end
        hs.status = "Progressing"
        hs.message = "Waiting for resource"
        return hs

4. Resource Quotas

# Prevent resource exhaustion
apiVersion: v1
kind: ResourceQuota
metadata:
  name: compute-quota
  namespace: development
spec:
  hard:
    requests.cpu: "10"
    requests.memory: 20Gi
    limits.cpu: "20"
    limits.memory: 40Gi
    persistentvolumeclaims: "10"

5. GitOps Workflow

1. Developer changes code
   ↓
2. CI builds image (myapp:abc123)
   ↓
3. CI updates manifest in GitOps repo
   ↓
4. ArgoCD detects change
   ↓
5. ArgoCD syncs to cluster
   ↓
6. Monitor health status

Migration Strategy

Migrating from kubectl apply

Phase 1: Capture Current State

# Export all resources
kubectl get all --all-namespaces -o yaml > current-state.yaml

# Clean up and organize
# Split into logical components

Phase 2: Create GitOps Repository

# Create repo structure
mkdir -p gitops-repo/apps/{app1,app2,app3}/base
mkdir -p gitops-repo/apps/{app1,app2,app3}/overlays/{dev,staging,prod}

# Move manifests
mv app1-manifests/* gitops-repo/apps/app1/base/

# Create kustomization files
# Commit to Git

Phase 3: Deploy ArgoCD

# Install ArgoCD
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

Phase 4: Onboard Applications Gradually

# Start with non-critical app
argocd app create app1-dev \
  --repo https://github.com/company/gitops-repo \
  --path apps/app1/overlays/dev \
  --dest-namespace dev

# Verify it works
argocd app sync app1-dev
argocd app get app1-dev

# Onboard remaining apps

Phase 5: Enable Auto-Sync

# Once confident, enable automation
argocd app set app1-dev --sync-policy automated

Conclusion

GitOps with ArgoCD transforms Kubernetes deployments from manual, error-prone processes into automated, auditable workflows. By making Git the source of truth, you gain:

Operational Benefits:

Consistent deployments across environments
Audit trail of all changes
Easy rollbacks via Git revert
Automated drift detection and correction

Team Benefits:

Developers self-service via Git commits
Platform engineers focus on platform, not deployments
Clear ownership (code owner = deployment owner)
Better collaboration through Git workflows

Key Takeaways:

Start simple. Deploy one application with auto-sync before building complex patterns.
Use Kustomize overlays. Best balance of simplicity and flexibility for most teams.
Automate carefully. Start with manual sync in production, enable auto-sync after confidence builds.
Monitor continuously. Set up alerts for OutOfSync and Degraded applications.
Secure properly. Use RBAC, sealed secrets, and project restrictions from day one.

GitOps is a journey, not a destination. Start with basics, measure success, and gradually adopt advanced features as your team's maturity grows.

Next Steps

Install ArgoCD in a test cluster and deploy your first application
Migrate one application from kubectl apply to GitOps
Set up monitoring with Prometheus and Grafana
Implement secrets management with Sealed Secrets or External Secrets Operator
Create an app-of-apps to manage multiple applications declaratively

Additional Resources

Questions about GitOps or ArgoCD? Share your implementation experiences in the comments below.

GitOps Workflow with ArgoCD

On this page

Prerequisites

Introduction

What is GitOps?

Core Principles

GitOps vs Traditional CI/CD

ArgoCD Architecture

Core Components

Key Concepts

Installation and Setup

Install ArgoCD on Kubernetes

Install ArgoCD CLI

Production Configuration

Basic Application Deployment

Repository Structure

Simple Deployment Manifest

Kustomize Overlays

ArgoCD Application Definition

Deploy Application via CLI

Multi-Environment Strategy

Pattern 1: Separate Git Branches

Pattern 2: Kustomize Overlays (Recommended)

Pattern 3: Helm with Values Files

App of Apps Pattern

ApplicationSet: Advanced Patterns

Pattern 1: Multiple Clusters

Pattern 2: Monorepo with Multiple Apps

Pattern 3: Pull Request Environments

CI/CD Integration

GitHub Actions Integration

GitLab CI Integration

Image Updater

Advanced Features

Progressive Delivery with Rollouts

Blue-Green Deployment

Resource Hooks

Sync Waves

Security Best Practices

Repository Credentials

Sealed Secrets

RBAC Configuration

Project Restrictions

Monitoring and Troubleshooting

Prometheus Metrics

Grafana Dashboard

Common Issues

Debugging Commands

Best Practices

1. Repository Structure

2. Sync Policies

3. Health Checks

4. Resource Quotas

5. GitOps Workflow

Migration Strategy

Migrating from kubectl apply

Conclusion

Next Steps

Additional Resources

Comments (0)

Leave a Comment