Github url

xg2xg

by jhuangtw-dev

jhuangtw-dev /xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

8.4K Stars 649 Forks Last release: Not found 418 Commits 0 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

A handy lookup table of similar technology and services to help ex-googlers survive the real world :) pull-requests very welcomed. Please do not list any confidential projects!

For a working example of (some) of these technologies integrated together, see: https://github.com/google/startup-os

See also: System Design Primer, The Hadoop Ecosystem Table, Map AWS services to GCP products

Technology

Core Technology

| Google Internal | Google External | Open Source / Real-World | | --------------- | ---------------------------------------- | ---------------------------------------- | | MapReduce | | Apache Hadoop, Spark | | Protocol Buffer | Protobuf | Cap'n Proto, Thrift, Avro, Amazon Ion, CBOR | | Stubby | gRPC | Thrift, Bolt | | Chubby | | Apache Zookeeper, etcd, HashiCorp Consul | | Goops / PubSub | | Apache Kafka, Apache Pulsar, Facebook LogDevice | | Dapper | | OpenTracing, jaeger, LightStep, Honeycomb | |

//base
| | abseil |

Infrastructure

| Google Internal | Google External | Open Source / Real-World | | -------------------- | ---------------------------------------- | ---------------------------------------- | | Borg | Kubernetes | Apache Mesos, Apache Aurora, HashiCorp Nomad | | GSLB | Cloud Load Balancing - Internal | AWS ELB, Istio, envoy, linkerd | GFE, Maglev, uberproxy | Cloud Load Balancing - HTTPS / External | AWS ALB, HAProxy, nginx, F5 | | uberproxy (sso proxy) | Identity-Aware Proxy | buzzfeed-sso, Pomerium | | data center hardware | open compute | | | Jupiter, Starblaze | | | | B4, Stargate, TE | | | | USPS, Andromeda | | |

Storage

| Google Internal | Google External | Open Source / Real-World | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | | GFS/Colossus | | HDFS, Ceph, GlusterFS, MooseFS | | BigTable | Cloud BigTable | PrestoDB, Cassandra, HBase, Accumulo, DynamoDB, ScyllaDB | | Spanner | Cloud Spanner | Vitess, CockroachDB, TiDB | | ColumnIO / Capacitor | | Apache Parquet | | sstable | levelDB | RocksDB | | zippy | Snappy | lz4 | | RecordIO | TFRecords | |

Services

| Google Internal | Google External | Open Source | SaaS | | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | ---------------------------------------- | | Dremel | BigQuery | Apache Drill, Presto, Spark(sort-of), | AWS Athena, Redshift Spectrum, Snowflake | | Dremel UI | | Redash, Metabase, Apache Superset | | | Search (Mustang, Alexandria) | | Elasticsearch, Solr, Lucene | algolia | | pubsub | pubsub | NATS.io, RabbitMQ, PubNub | AWS SQS/SNS, AWS AppSync | | Flume (Java) | Apache Beam | Apache Crunch | | | MillWheel | Cloud Dataflow | Apache Flink | | | Colab | Colaboratory | Jupyter | | | PLX | Google Data Studio | | Mode |

DevOps

| Google Internal | Google External | Real-World | | ------------------------------- | ---------------------------------------- | ---------------------------------------- | | Assimilator | | Dominator | | Blaze | Bazel | Buck, Pants, please.build, Blade, recc | | Oncall | | PagerDuty, OpsGenie, VictorOps | | varz/borgmon/monarch | | Datadog, Prometheus, M3, librato, newrelic, skylight, scout, Scotty/tricorder, netdata, bosun, also this and this | | Viceroy | | Grafana | | Exception/Error Tracking (??) | | Sentry.io, Raygun.io, Rollbar, Honeybadger, Airbrake, OverOps, ELK stack | | styleguides | google styleguides | PEP-8, HoundCI auto-style-reviewer | | Blaze/Forge/TAP/BuildCop | Cloud Build | buildkite, CircleCI, travis, jenkins, gitlabCI, github, Drone | | Forge/ObjFS | | EngFlow | | Sandman(test env)/Guitar | | | | Sisyphus / Rapid | | Spinnaker, lambdaCD, screwdriver.cd, CodeShip, shipit-engine, GoCD, AWS CodeDeploy, Capistrano, Fabric, ConcourseCI, samson | | borg / borgcfg / gcl | Jsonnet, Cue | AWS Cloudformation, Puppet, Chef, Salt, Ansible, Terraform, kubecfg, pulumi | | logging, analog | StackDriver | logstash, fluentd, PaperTrail, cernan | | CodeSearch, Grimoire | Zoekt | Sourcegraph, OpenGrok | | Critique, Gerrit, Mondrian etc. | Gerrit | Reviewable , Phabricator | | cider | | Eclipse Che, Cloud9, gitpod.io, Coder, Code-Server (VSCode in a Tab)| | buganizer | Google Issue Tracker | JIRA, bugzilla, github issues | | ToTT | Google Test Blog | Increment | | Copybara / MOE | Copybara, MOE | | | workflow/dependency management | | Luigi, Airflow, digdag, Pachyderm, Dask | | ErrorProne | ErrorProne | SpotBugs, FindBugs | | dapper (distributed trace) | stackdriver trace | zipkin, opentracing, jaeger | | C++ Tips of the Week | Abseil C++ Tips of the Week | |

Security

| Google Internal | Google External | Open Source | | -------------------------------- | --------------- | ---------------------------------------- | | prodaccess/LOAS | | Keymaster | | prod secrets/identity management | | chamber, knox, SPIFFE |

IT / Operations

| Google Internal | Real-World | | ---------------------------------------- | ---------------------------------------- | | software engineering at google | | | valentine | Vault, 1Password, Lastpass, pass | | OWNERS files in repo | github CODEOWNERS | | snippets | Khan/snippets | | SnipIt | recordit, CloudApp, dropbox screenshots | | edge, people ops training | LifeLabs | | googlegeist | Culture Amp, humu, tinypulse, peakon | | HRIS/ERP | Namely, BambooHR, Workday | | stuff (SaaS IT management) | productiv, intello, zylo | | stuff (Device Management) | Fleetsmith, jamf, rippling IT | | device security monitoring | Red Canary | | beyondcorp | beyondcorp | | go/ links | golinks, go, Goat, trotto | | google3 philosophy | innersource, monorepo | | doing code review | code review | | safely sharing 1-time secrets | croc, onetimesecret, privatebin | | messaging | mattermost, Slack, gchat | | peer bonus | bonus.ly, zestful | | kudos | heytaco, slack ++ bot |

also check out xoogler.co, which organizes events, slack channels etc

disclaimer: I'm not affiliated with any of the technologies mentioned above.

disclaimer: I left Google a number of years ago so some of the naming might be dated

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.