Once AerSight is installed and set up in your GCP environment, data is streamed automatically on a periodic basis (every 5 minutes).
Data is provided according to a structure as follows.
{
"data": "<payload in CSV format, see below>",
"id": "df88e808-f7ba-4486-9c41-f3b8cd78a23f",
"source": "aeris/acp/ipstream",
"specversion": "1.0",
"type": "ipstream.data",
"datacontenttype": "text/csv",
"time": "2021-03-30T09:18:09.695729Z"
}
Payload
format:
device_id, imsi, msisdn, primary_min, meid, technology, session_id, session_start_time, source_ip,
network_protocol, destination_ip, uplink_data, downlink_data, received_time, eventdate, matched_ip,
application_protocol, source_port, destination_port, product_name, destination_host
In GCP, you can view the BigQuery DataSet, aersight_acp_ds, click the device table, and click Preview to see the data.
Data Monitoring
It is recommended that you enable monitoring on the following GCP services to proactively
identify any issues with the data stream.
- PubSub: You can apply monitoring on the Google Cloud Metrics named subscription/num_undelivered_messages. This allows you to monitor the number of unacknowledged messages in a subscription.
- DataFlow: You can apply monitoring on the data flow jobs:
- System Lag (job/system_lag)
- VM Resource utilization (CPU, Memory) (job/current_num_vcpus,job/gpu_memory_utilization)
- Job Failed / Status: Alert if it is in the failed state (job/is_failed)
- PubSub – read latency (job/pubsub/read_latencies)
0 Comments