Need help with snowplow?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

snowplow
5.5K Stars 1.1K Forks Apache License 2.0 6.2K Commits 101 Opened issues

Description

Cloud-native web, mobile and event analytics, running on AWS and GCP

Services available

!
?

Need anything else?

Contributors list

Snowplow

License

Snowplow logo

Snowplow is an enterprise-strength marketing and product analytics platform. It does three things:

  1. Identifies your users, and tracks the way they engage with your website or application
  2. Stores your users' behavioural data in a scalable "event data warehouse" you control: Amazon Redshift, Google BigQuery, Snowflake or Elasticsearch
  3. Lets you leverage the biggest range of tools to analyze that data, including big data tools (e.g. Spark) via EMR or more traditional tools e.g. Looker, Mode, Superset, Re:dash to analyze that behavioural data

To find out more, please check out Snowplow website and the docs website.

If you wish to get everything setup and managed for you, you can take a look at our commercial offer, Snowplow Insights.

Snowplow technology 101

The repository structure follows the conceptual architecture of Snowplow, which consists of six loosely-coupled sub-systems connected by five standardized data protocols/formats:

architecture

To briefly explain these six sub-systems:

  • Trackers fire Snowplow events. Currently we have 12 trackers, covering web, mobile, desktop, server and IoT
  • Collector receives Snowplow events from trackers. Currently we have one official collector implementation with different sinks: Apache Kafka, Amazon Kinesis, NSQ
  • Enrich cleans up the raw Snowplow events, enriches them and puts them into storage. Currently we have several implementations, built for different environments (GCP, AWS, Apache Kafka) and one core library
  • Storage is where the Snowplow events live. Currently we store the Snowplow events in a flatfile structure on S3, and in the Redshift, Postgres, Snowflake and BigQuery databases
  • Data modeling is where event-level data is joined with other data sets and aggregated into smaller data sets, and business logic is applied. This produces a clean set of tables which make it easier to perform analysis on the data. We have data models for Redshift and Looker
  • Analytics are performed on the Snowplow events or on the aggregate tables.

For more information on the current Snowplow architecture, please see the Technical architecture.

About this repository

This repository used to be an umbrella repository for all loosely-coupled Snowplow components. However, since June 2020 all components have been extracted into their dedicated repositories (more info here) and this repository serves as an entry point for OSS users and historical artifact.

Components that have been extracted to their own repo are still here as git submodules.

Please use directly the repo of a component to report issues and create PRs:

Trackers

Collector

Enrich

Loaders

Testing

Parsing enriched event

Bad rows

Need help?

We want to make it super-easy for Snowplow users and contributors to talk to us and connect with each other, to share ideas, solve problems and help make Snowplow awesome. Here are the main channels we're running currently, we'd love to hear from you on one of them:

Discourse

This is for all Snowplow users: engineers setting up Snowplow, data modelers structuring the data and data consumers building insights. You can find guides, recipes, questions and answers from Snowplow users including the Snowplow team.

We welcome all questions and contributions!

Twitter

@SnowplowData for official news or @SnowplowLabs for engineering-heavy conversations.

GitHub

If you spot a bug, then please raise an issue in the GitHub repo of this component. Likewise if you have developed a cool new feature or an improvement, please open a pull request, we'll be glad to integrate it in the codebase!

If you want to brainstorm a potential new feature, then Discourse is probably a better place to start.

Email

[email protected]

If you want to talk directly to us (e.g. about a commercially sensitive issue), email is the easiest way.

Copyright and license

Snowplow is copyright 2012-2020 Snowplow Analytics Ltd.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.