How to look at Pittsburgh arrest data using an API - Beginner-friendly

 Find the data you want to access

The Western PA Regional Data Center (WPRDC) is an awesome resource with lots of Pittsburgh-related data. I wanted to access Pittsburgh city arrest data. 

After you find your data file, you'll need to find the file's resource id. It's a hash, a long string of letters & numbers with four hyphens. There's a few places this is located. If you download a sample file, the name of that file is the id. The resource id is also the last section of the page's URL. If you're visiting this webpage:

https://data.wprdc.org/datastore/dump/e03a89dd-134a-4ee8-a2bd-62c40aeebc6f

then your resource ID is  

e03a89dd-134a-4ee8-a2bd-62c40aeebc6f 

 

Build your request URL

WPRDC uses DataStore to house its data. You can ask WPRDC to send you data by requesting it in a URL format the database understands, an API. To make a request, we'll use the datastore_search call. Here's the base of the URL:

https://data.wprdc.org/api/3/action/datastore_search?

Now we need to tell the database which file we're looking for. Add your request id:

https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f

Since we're just getting started still, let's add a limit on the amount of data WPRDC will send us at once. That way we don't accidentally request huge data files, wasting resources for both us and WPRDC. I'll start with a limit of 5 rows of data. Once we know our program is working, we can come back later and remove the limit. 

https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f&limit=5  

There's lots of other helpful tricks you can use when building your URL. For example, you can add filters to reduce the amount of irrelevant data you receive. The datastore_search documentation explains more. For now, just hold on to that URL.

Install Python and a few libraries

Install Python 3. 

Install requests and pandas. The easiest way is to use pip. 

pip install requests

pip install pandas 

(Optional) I like to use Jupyter notebooks to mess around with tabular data. Consider installing Jupyter Lab and running your code in a notebook. 

pip install notebook 

jupyter notebook 

Request your data using Python

Open a new Python file or notebook, and import your libraries:

import requests

import pandas as pd

 

Paste in the request URL we built earlier:

url = https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f&limit=5 

Ask (request) WPRDC to send you that data, then turn it into a readable format (JSON). 

resp = requests.get(url).json()

The response contains a bunch of extra metadata that we don't need right now. So let's grab the meaty part of the response ("result") and pull out the actual data ("records").  We'll use pandas to turn that into a nice spreadsheet table (a "DataFrame"). 

data = pd.DataFrame(resp['result']['records'])

data (if you're using a notebook)

display(data.to_string()) (if you're not using a notebook) 

 

A note on API politeness

Every time you run requests.get(url), you're connecting to the web, asking WPRDC to send you data, and downloading the response. When you're using an API, try to be conscientious about the frequency and size of the requests you're making. Many databases will enforce a limit on the number of requests you make in a day, and some will even ban your IP address if they think you're trying to abuse their servers with tons of spammy requests. I didn't find any documentation about the API limits for WPRDC, but it's still best-practice to be intentional about how you design your code to only request new data when you actually need it.  

 

This tutorial was written in August 2023. 

A hack to make it easier for QA Testers in Twine (Sugarcube 2)

For the game OnlyBans, we got a wonderful accessibility audit done by the Disability Sexuality Access Network. We also periodically have non-technical QA volunteers and playtesters give feedback on the game. I don't really want them to have to learn a bunch of new tools to give effective feedback and bug reports. 

Goals:

• Testers can easily jump to/from/back to passages
• Testers can easily reference the title of the passage they're looking at
• Testers don't have to learn Twine or have overwhelming debug tools
• Don't insert new content on the screen. (e.g., I can't just add the title to the top of every page
• Make "QA Mode" something I can easily toggle on/off

To achieve these goals, I relied on adding content through the UI Bar. If your game doesn't have the Sugarcube UI bar, or if your game has complex path logic or a nonlinear history, these tactics might not be helpful for you.

1. Make a QA Passage

This passage will be a list of all the passage titles.

As far as I can tell*, there's no Sugarcube API to directly reference all the passage objects. So I instead searched all the Story objects to find objects with a non-empty passage title. Then, print that array as a link. 

!! All Pages

<<silently>>

  <<set $allPassages to []>>

  <<script>>

    var $anyRegExp = new RegExp('');

    var $passageTitles = Story.lookupWith(function (_p) {

        return $anyRegExp.test(_p.title);

        }).map(_psg => _psg.title);

    state.variables.allPassages = $passageTitles;

  <</script>>

<</silently>>

<<for _i to 0; _i lt $allPassages.length; _i++>>

<<link $allPassages[_i] $allPassages[_i]>><</link>>

<</for>>

QA Passage example 

With a few more lines of code, you could probably use the Story APIs to filter out the behind-the-scenes passages with Special Names. For example, play testers probably don't need to see the StoryInit passage listed here. 

Sometimes I'll manually add a few passages of special note. For example, during accessibility testing, I listed out a few specific passages that had complex interactions and might need special attention.

2. Make a QA mode boolean flag

In your game javascript, give yourself the nice gift of having a global variable that controls the logic of whether you want to play in the game in QA mode. 

var $isQATestMode = true;

State.setVar("$isQATestMode", $isQATestMode);

3. Add your new QA Passage to the UI Sidebar

In your StoryMenu passage (make one if you don't have one already), add a handy link to your QA Test passage. Here's what ours looks like:

[[How to Play]]

[[About]]

[[Credits]]

<<if $isQATestMode >>[[QA Test]]<</if>>

4. Display the passage name in the sidebar

In your StorySubtitle passage (make one if you don't have one already), display the name of the passage the player is currently on. That way, they can reference the correct passage if they're making a bug report. 

<<if $isQATestMode>>Current Passage: <<print passage()>><</if>>

5. Enable jumping backward & forward in time

In your game javascript file, use the handy Sugarcube Config API to make it so playtesters can go back in time. Usually I don't want my players to be able to do that.  This setting automagically adds a back button and history lightning bolt in the UI Sidebar. 

if($isQATestMode){
console.log("QA Test Mode Enabled");

Config.passages.descriptions = true;  //sets descriptions for Jump To

Config.history.controls = true; //enables back button

}
else{
console.log("NOT QA");
Config.history.controls = false; //disable backward and forward buttons

}

Screenshot with QA Mode OFF

Screenshot with QA Mode ON

What are other tricks you've used to make it easier to collaborate on Twine games? I'm especially curious for those of you with non-technical collaborators. 

*I'm usually wrong about this sort of thing, to be fair.