mixpanel - MixPanel JQL Collector

This collector is a thin wrapper around the Mixpanel JQL API, converting JQL query results into Anodot metrics. The user writes the JQL Query so that it returns an array of objects of the form:

{
    "key": [ "timestamp", "dimension_1_value", "dimension_2_value", "...." ],
    "value": "some_number"
}

Note

this collector supports only a single metric per query

Account-level properties

• mixpanel_secret - The User’s Mixpanel API secret token.
• mixpanel_endpoint (optional) - an alternative endpoint.

Query-level properties

• dimension_defs - this collector’s dimensions are taken from the JQL Query result key, starting from the second element.
• metric_defs - An array containing a single metric definition of the metric to be sent to Anodot.
• script - the JQL script to be run by MixPanel to produce the results.
• extra_params - parameters which are passed as part of the params in the JQL - see Mixpanel docs.

Queries using the simple.jql script

The mixpanel collector allows you to use any predefined JQL script according to your preferences.

Additionally, we included a script called simple.jql for convenience, to address the simplest cases.

The simple.jql script performs a query on the selected time range, returning the metric grouped according to the settings in the anodot-collect-conf.json.

an example of anodot-collect-conf.json

{
    "queries": {
        "ErrorOcurred_by_errorcode_and_contextcode": {
            "script": { "file" : "simple.jql" },
            "extra_params": {
                "time_resolution":  "@eventHour",
                "event_selectors":  [ {"event": "ErrorOcurred"} ],
                "reducer":          "mixpanel.reducer.count()",
                "utc_offset":       "-8"
            },
            "dimension_defs":  [
                { "dimension": "name" },
                { "dimension": "properties.errorcode" },
                { "dimension": "properties.contextcode" }
            ],
            "metric_defs": [ { "metric": "event_count", "target_type": "counter" } ]
        }
    }
}

Explanation:

• script - the simple.jql script
• extra-params - defines the parameters to be passed to the JQL script in the params object:
  • time_resolution - either @eventDate or @eventHour for aggregation.
  • reducer - type of reducer to use - see mixpanel JQL docs.
  • utc_offset - Important: UTC offset of your mixpanel project timezone ( The timezone of your project appears in Mixpanel’s Dashboard under Project Settings ). In the above example, “-8” means the project timezone is Pacific Time.
• dimension_defs - the list of dimensions is passed into the the JQL script as part of the params object. It may be used to dynamically create a query. The names to use here depend on how the JQL Script interprets them. Also, dimension_defs is used in building the metric name for Anodot.

Using other JQL Scripts:

The Mixpanel collector uses the Mixpanel JQL API in order to collect metrics. The user needs to write the JQL Query so that it returns an array of objects of the form:

{
    "key": [ "timestamp", "dimension_1_value", "dimension_2_value", "...." ],
    "value": "some_number"
}

And accordingly configure the dimension_defs so that the returned key array corresponds to the dimension_defs dimensions.

Account-level properties

• mixpanel_secret - The User’s Mixpanel API secret token. See also the -M command-line parameter.

Query-level properties

• dimension_defs - this collector’s dimensions are taken from the JQL Query result key, starting from the second element.
• metric_defs - An array containing a single metric definition of the metric to be sent to Anodot.
• script - the JQL script to be run by MixPanel to produce the results.
• extra_params - a JSON object to be passed to the JQL script as the params object ( see the MixPanel JQL API Reference.

Note

Although metric_defs is a list, this collector supports only a single metric per query.

Command-line parameters

usage: anodot-collect.py mixpanel [-h] [-s START_DATE] [-e END_DATE]
                                  [-M MIXPANEL_SECRET] [-w WORK_DIR]
                                  [-E ENDPOINT] [-T ANODOT_API_TOKEN] [-V VER]
                                  [-a] [-J] [-d] [-p] [-D API_DELAY]
                                  [-C API_CHUNK]
                                  [--producer-concurrency PRODUCER_CONCURRENCY]
                                  [--anodot-api-concurrency ANODOT_API_CONCURRENCY]
                                  [-r] [-t]

optional arguments:
  -h, --help            show this help message and exit
  -s START_DATE, --start-date START_DATE
                        Value to be passed as params.start_date to your JQL
                        query in format YYYY-MM-DD (default: yesterday)
  -e END_DATE, --end-date END_DATE
                        Value to be passed as params.end_date to your JQL
                        query in format YYYY-MM-DD (default: yesterday)
  -M MIXPANEL_SECRET, --mixpanel-secret MIXPANEL_SECRET
                        the mixpanel secret key to use for queries
  -w WORK_DIR, --work-dir WORK_DIR
                        working directory ( logs saved there )
  -E ENDPOINT, --endpoint ENDPOINT
                        Anodot API endpoint: `poc` or `production`
  -T ANODOT_API_TOKEN, --anodot-api-token ANODOT_API_TOKEN
                        Anodot API endpoint: `poc` or `production`
  -V VER, --ver VER     data version number to send ( unless specified here or
                        in the config, ver=1 )
  -a, --save-anodot-csv
                        write a CSV file with the Anodot format
  -J, --save-anodot-json
                        write a JSON file with the same format being sent to
                        the Anodot API
  -d, --debug           Print verbose debug output
  -p, --production
  -D API_DELAY, --api-delay API_DELAY
                        Anodot API: delay between requests
  -C API_CHUNK, --api-chunk API_CHUNK
                        Anodot API: max. number of metrics to send per request
  --producer-concurrency PRODUCER_CONCURRENCY
                        Number of concurrent processes to use for producers
  --anodot-api-concurrency ANODOT_API_CONCURRENCY
                        Number of concurrent processes to use for sending to
                        anodot

Downloads

• The simple.jql file: (Download).
• An elaborate anodot-collect-conf.json example (Download).