devops-tools cheatsheet googlers lookup-table infrastructure management
Need help with xg2xg?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.
jhuangtw

Description

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

8.7K Stars 666 Forks 457 Commits 17 Opened issues

Services available

Need anything else?

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products

Technology

Core Technology

| Google Internal | Google External | Open Source / Real-World | | --------------- | ---------------------------------------- | ---------------------------------------- | | MapReduce | | Apache Hadoop, Spark | | Protocol Buffer | Protobuf | Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR | | Stubby | gRPC | Thrift, Bolt | | Chubby | | Apache Zookeeper, etcd, HashiCorp Consul | | Goops / PubSub | | Apache Kafka, Apache Pulsar, Facebook LogDevice | |

//base
| | abseil |

Infrastructure

| Google Internal | Google External | Open Source / Real-World | | -------------------- | ---------------------------------------- | ---------------------------------------- | | Borg | Kubernetes | Apache Mesos, Apache Aurora, HashiCorp Nomad | | GSLB | Cloud Load Balancing - Internal | AWS ELB, Istio, envoy, linkerd | GFE, Maglev, uberproxy | Cloud Load Balancing - HTTPS / External | AWS ALB, HAProxy, nginx, F5 | | uberproxy (sso proxy) | Identity-Aware Proxy | buzzfeed-sso, Pomerium | | data center hardware | open compute | | | Jupiter, Starblaze | | | | B4, Stargate, TE | | | | USPS, Andromeda | | | | ESDN | | Faucet

Storage

| Google Internal | Google External | Open Source / Real-World | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | | GFS/Colossus | | HDFS, Ceph, GlusterFS, MooseFS | | BigTable | Cloud BigTable | PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB | | Spanner | Cloud Spanner | Vitess, CockroachDB, TiDB | | ColumnIO / Capacitor | | Apache Parquet | | sstable | levelDB | RocksDB, pebble | | zippy | Snappy | lz4 | | RecordIO | TFRecords | |

Services

| Google Internal | Google External | Open Source | SaaS | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | | Dremel | BigQuery | Apache Drill, Presto, Spark(sort-of), | AWS Athena, Redshift Spectrum, Snowflake | | Dremel UI | | Redash, Metabase, Apache Superset | | | Search (Mustang, Alexandria) | | Elasticsearch, Solr, Lucene | algolia | | pubsub | pubsub | NATS.io, RabbitMQ, PubNub | AWS SQS/SNS, AWS AppSync | | Flume (Java) | Apache Beam | Apache Crunch | | | MillWheel | Cloud Dataflow | Apache Flink | | | Colab | Colaboratory | Jupyter | | | PLX | Google Data Studio | | Mode |

DevOps

| Google Internal | Google External | Real-World | | ------------------------------- | ---------------------------------------- | ---------------------------------------- | | Assimilator | | Dominator | | Blaze | Bazel | Buck, Pants, please.build, Blade, recc | | Oncall | | PagerDuty, OpsGenie, VictorOps | | varz/borgmon/monarch | | Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this | | Viceroy | | Grafana | | Exception/Error Tracking (??) | | Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack | | styleguides | google styleguides | PEP-8, HoundCI auto-style-reviewer | | Blaze/Forge/TAP/BuildCop | Cloud Build | buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone | | Forge/ObjFS | | EngFlow | | Sandman(test env)/Guitar | | | | Sisyphus / Rapid | | Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson | | borg / borgcfg / gcl | Jsonnet, Cue | AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi | | logging, analog | StackDriver | logstash, fluentd, PaperTrail, cernan | | CodeSearch, Grimoire | Zoekt | Sourcegraph, OpenGrok | | Critique, Gerrit, Mondrian etc. | Gerrit | Reviewable , Phabricator | | cider | | Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)| | buganizer | Google Issue Tracker | JIRA, bugzilla, github issues | | ToTT | Google Test Blog | Increment | | Copybara / MOE | Copybara, MOE | | | workflow/dependency management | | Luigi, Airflow, digdag, Pachyderm, Dask | | ErrorProne | ErrorProne | SpotBugs, FindBugs | | Dapper | stackdriver trace | zipkin, opentracing, jaeger, LightStep, Honeycomb | | C++ Tips of the Week | Abseil C++ Tips of the Week | |

Security

| Google Internal | Google External | Open Source | | -------------------------------- | --------------- | ---------------------------------------- | | prodaccess/LOAS | | Keymaster | | prod secrets/identity management | | chamber, knox, SPIFFE |

IT / Operations / Misc

| Google Internal | Real-World | | ---------------------------------------- | ---------------------------------------- | | software engineering at google | | | valentine | Vault, 1Password, Lastpass, pass | | OWNERS files in repo | github CODEOWNERS | | snippets | Khan/snippets | | SnipIt | recordit, CloudApp, dropbox screenshots, Snippyly | | edge, people ops training | LifeLabs | | googlegeist | Culture Amp, humu, tinypulse, peakon | | HRIS/ERP | Namely, BambooHR, Workday | | stuff (SaaS IT management) | productiv, intello, zylo | | stuff (Device Management) | Fleetsmith, jamf, rippling IT | | device security monitoring | Red Canary | | beyondcorp | beyondcorp | | go/ links | golinks, go, Goat, trotto | | google3 philosophy | innersource, monorepo | | doing code review | code review | | safely sharing 1-time secrets | croc, onetimesecret, privatebin | | messaging | mattermost, Slack, gchat | | peer bonus | bonus.ly, zestful | | kudos | heytaco, slack ++ bot | | tech talks | TechTalks @ Google | | talks | Talks @ Google | | g3doc | writethedocs, docs-as-code |

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.