Only Org Admins can download this data via the AWS CLI.
To get the data:
Generate a new api token for s3_labs
Make an api call with:
curl --location --request GET 'https://api.vocareum.com/api/v2/orgs/{{orgid}}/s3_labs' --header 'Authorization: Token {{token}}'This returns temporary credentials for AWS S3 access for one hour.
Run the following commands in your terminal:
export AWS_ACCESS_KEY_ID=ASIAIOSFODNN7EXAMPLE
export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
export AWS_SESSION_TOKEN=AQoDYXdzEJr...
aws s3 sync s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/ .
(the values of AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY , AWS_SESSION_TOKEN are given in the response from the previous api call)
Directory Structure
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/lab_sessions/lab_sessions_y-m-d.csv
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/cloud_spend/aws_cost_y-m-d.csv
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/service_spend/service_spend_y-m.csv
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/grades/course{{courseid}}/grades.csv
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/course-asn_mapping/{{course/asn}}_mapping.csv
s3://vocareum-us-west-2-lab-analytics/org{{orgid}}/enrollment/course{{courseid}}/enrollment.csv
Lab Sessions, AWS Cost and Service Spend
The service that generates these files runs every 6 hours and grabs all of the previous day's data (UTC time). Lab Sessions and AWS Cost are daily files. While Service Cost is a monthly file.
lab_sessions
Records all lab types (VM, Container, Cloud).
"User ID", "User Email", "Course ID", "Course Name", "Assignment ID", "Assignment Name", "Part ID", "Part Name", "Lab Type", "Start Time", "End Time", "Duration (min)", Log, Tags, "Launch Time", "Init Time", "Launch To Ready Duration (mins)", "Concurrent Labs Count", "Disconnects", "Latency Time (ms)"
Launch Time: when "Start Lab" is clicked
Start Time: when EC2 instance is started
Init Time: when lab is actually ready to use
Launch To Ready Duration (mins): time it takes from launch time to init time
Concurrent Labs Count: number of labs already started when user starts lab
Disconnects: number of times user gets disconnected
ex:
1731xxx, user@vocareum.com, 45686, "Kevin 2021", 760081, lab-04-api, 760084, assignment-part, Cloud,"Feb-23-2022 9:40:20 pm UTC", "Feb-23-2022 11:40:22 pm UTC", 120.03333333333," START successful; CFN creation successful; END successful (Session Timer Expires)",,"Jun-01-2022 9:14:57 pm UTC", "Jun-01-2022 9:15:02 pm UTC", 0.083333333333333, 180
aws_cost
Spend accumulated per account.
"AWS Account ID", "User ID", "User Email", "Part ID", "Part Name", Cost, "File Updated Time"
ex:
8622xxxx, 1731xxxx, user@vocareum.com, 760084, assignment-part, 0.038598, 1645823566
service_spend
Cost accumulated for a particular AWS service per account.
"AWS Account ID", "User Id", "User Email", "Course ID", "Part ID", Service, Cost
ex:
862252xxx, 173xxxx, user@vocareum.com, 45686,760084, "Amazon Elastic Compute Cloud", 0.037481
Assignment Mapping, Course Mapping, Grades, and Enrollment
The service that generates mapping, grades, and enrollment data runs every 24 hours
asn_mapping
List of assignments in a course and copies of those assignments.
org_id, course_id, assignment_id, assignment_name, part_id, part_name, parent_assignment_id, parent_part_id
ex:
9xxx, 1020, 190155, "AWS Sandbox", 190156, Lab, 50833, 50834
course_mapping
List of courses in the org and clones of those courses.
org_id, org_name, course_id, course_name, create_method, parent_course_id
ex:
9xxx, "Example Org", 4051, "Example CS 17 - 1", "Clone by Reference", 10x
grades
courseid, coursename, asnid, asnname, ispeerreviewon, maxscore, userid, partnerid, asnscore, isoverride, partid, partname, partscore, rubricid, rubricname, rubricscore, graderid
ex:
20154, "Test Course", 412074, "Makeup HW 1 (grading script)", 0, 14, 9260xx, 10, 0, 412075, "Makeup HW 1", 10, 551636, Score, 10, 926xx
enrollment
userid, courseid, role, name, email, client_key, client_userid, dropped, organization_terms_agreed, organization_terms_agreed_date_utc
ex:
1731xxx, 20154, student, user@vocareum.com, demokey, 9999999, 0,,