How to look at Pittsburgh arrest data using an API - Beginner-friendly

 Find the data you want to access

The Western PA Regional Data Center (WPRDC) is an awesome resource with lots of Pittsburgh-related data. I wanted to access Pittsburgh city arrest data. 

After you find your data file, you'll need to find the file's resource id. It's a hash, a long string of letters & numbers with four hyphens. There's a few places this is located. If you download a sample file, the name of that file is the id. The resource id is also the last section of the page's URL. If you're visiting this webpage:

https://data.wprdc.org/datastore/dump/e03a89dd-134a-4ee8-a2bd-62c40aeebc6f

then your resource ID is  

e03a89dd-134a-4ee8-a2bd-62c40aeebc6f 

 

Build your request URL

WPRDC uses DataStore to house its data. You can ask WPRDC to send you data by requesting it in a URL format the database understands, an API. To make a request, we'll use the datastore_search call. Here's the base of the URL:

https://data.wprdc.org/api/3/action/datastore_search?

Now we need to tell the database which file we're looking for. Add your request id:

https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f

Since we're just getting started still, let's add a limit on the amount of data WPRDC will send us at once. That way we don't accidentally request huge data files, wasting resources for both us and WPRDC. I'll start with a limit of 5 rows of data. Once we know our program is working, we can come back later and remove the limit. 

https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f&limit=5  

There's lots of other helpful tricks you can use when building your URL. For example, you can add filters to reduce the amount of irrelevant data you receive. The datastore_search documentation explains more. For now, just hold on to that URL.

Install Python and a few libraries

Install Python 3. 

Install requests and pandas. The easiest way is to use pip. 

pip install requests

pip install pandas 

(Optional) I like to use Jupyter notebooks to mess around with tabular data. Consider installing Jupyter Lab and running your code in a notebook. 

pip install notebook 

jupyter notebook 

Request your data using Python

Open a new Python file or notebook, and import your libraries:

import requests

import pandas as pd

 

Paste in the request URL we built earlier:

url = https://data.wprdc.org/api/3/action/datastore_search?resource_id=e03a89dd-134a-4ee8-a2bd-62c40aeebc6f&limit=5 

Ask (request) WPRDC to send you that data, then turn it into a readable format (JSON). 

resp = requests.get(url).json()

The response contains a bunch of extra metadata that we don't need right now. So let's grab the meaty part of the response ("result") and pull out the actual data ("records").  We'll use pandas to turn that into a nice spreadsheet table (a "DataFrame"). 

data = pd.DataFrame(resp['result']['records'])

data (if you're using a notebook)

display(data.to_string()) (if you're not using a notebook) 

 

A note on API politeness

Every time you run requests.get(url), you're connecting to the web, asking WPRDC to send you data, and downloading the response. When you're using an API, try to be conscientious about the frequency and size of the requests you're making. Many databases will enforce a limit on the number of requests you make in a day, and some will even ban your IP address if they think you're trying to abuse their servers with tons of spammy requests. I didn't find any documentation about the API limits for WPRDC, but it's still best-practice to be intentional about how you design your code to only request new data when you actually need it.  

 

This tutorial was written in August 2023. 

A hack to make it easier for QA Testers in Twine (Sugarcube 2)

For the game OnlyBans, we got a wonderful accessibility audit done by the Disability Sexuality Access Network. We also periodically have non-technical QA volunteers and playtesters give feedback on the game. I don't really want them to have to learn a bunch of new tools to give effective feedback and bug reports. 

Goals:

• Testers can easily jump to/from/back to passages
• Testers can easily reference the title of the passage they're looking at
• Testers don't have to learn Twine or have overwhelming debug tools
• Don't insert new content on the screen. (e.g., I can't just add the title to the top of every page
• Make "QA Mode" something I can easily toggle on/off

To achieve these goals, I relied on adding content through the UI Bar. If your game doesn't have the Sugarcube UI bar, or if your game has complex path logic or a nonlinear history, these tactics might not be helpful for you.

1. Make a QA Passage

This passage will be a list of all the passage titles.

As far as I can tell*, there's no Sugarcube API to directly reference all the passage objects. So I instead searched all the Story objects to find objects with a non-empty passage title. Then, print that array as a link. 

!! All Pages

<<silently>>

  <<set $allPassages to []>>

  <<script>>

    var $anyRegExp = new RegExp('');

    var $passageTitles = Story.lookupWith(function (_p) {

        return $anyRegExp.test(_p.title);

        }).map(_psg => _psg.title);

    state.variables.allPassages = $passageTitles;

  <</script>>

<</silently>>

<<for _i to 0; _i lt $allPassages.length; _i++>>

<<link $allPassages[_i] $allPassages[_i]>><</link>>

<</for>>

QA Passage example 

With a few more lines of code, you could probably use the Story APIs to filter out the behind-the-scenes passages with Special Names. For example, play testers probably don't need to see the StoryInit passage listed here. 

Sometimes I'll manually add a few passages of special note. For example, during accessibility testing, I listed out a few specific passages that had complex interactions and might need special attention.

2. Make a QA mode boolean flag

In your game javascript, give yourself the nice gift of having a global variable that controls the logic of whether you want to play in the game in QA mode. 

var $isQATestMode = true;

State.setVar("$isQATestMode", $isQATestMode);

3. Add your new QA Passage to the UI Sidebar

In your StoryMenu passage (make one if you don't have one already), add a handy link to your QA Test passage. Here's what ours looks like:

[[How to Play]]

[[About]]

[[Credits]]

<<if $isQATestMode >>[[QA Test]]<</if>>

4. Display the passage name in the sidebar

In your StorySubtitle passage (make one if you don't have one already), display the name of the passage the player is currently on. That way, they can reference the correct passage if they're making a bug report. 

<<if $isQATestMode>>Current Passage: <<print passage()>><</if>>

5. Enable jumping backward & forward in time

In your game javascript file, use the handy Sugarcube Config API to make it so playtesters can go back in time. Usually I don't want my players to be able to do that.  This setting automagically adds a back button and history lightning bolt in the UI Sidebar. 

if($isQATestMode){
console.log("QA Test Mode Enabled");

Config.passages.descriptions = true;  //sets descriptions for Jump To

Config.history.controls = true; //enables back button

}
else{
console.log("NOT QA");
Config.history.controls = false; //disable backward and forward buttons

}

Screenshot with QA Mode OFF

Screenshot with QA Mode ON

What are other tricks you've used to make it easier to collaborate on Twine games? I'm especially curious for those of you with non-technical collaborators. 

*I'm usually wrong about this sort of thing, to be fair. 

How to display media with relative links in Twine play testing mode

Twine is great, and the Twine 2 UI is a fantastic little tool for developing nonlinear stories. But as of this writing, it's great for text and not-so-great for other media. You can easily embed media using html, but it's not easy to work with as you're developing or testing your game. One frustrating note from the Twine documentation:

If you are using relative links for your multimedia, you will need to publish your story to the correct location on your computer so that the references point to the right place. If you have an image tag whose source property is “myimage.jpeg”, for example, then your published file must be placed in the same folder as the file myimage.jpeg. Learn more about relative links

If you are using relative links, testing or playing your story inside Twine will unfortunately not work.

I was using many local, relative links for images and video as I was working on OnlyBans and got annoyed enough by this to make a temporary solution for myself. To use this, you need a file syncing tool. 

1. Make a media/ directory in the same location that your Twine story lives. This is likely your Twine/Stories directory. Drop your images here: Twine/Stories/images/myimage.jpg. (This step is largely for convenience.)
2. Open your story and add an image to a story passage using an HTML tag such as <img> or <video>
3. Test your story. You'll notice that instead of images, you'll get a little default 'missing image' icon. 
4. Open the developer view tool for your browser. Find your image in the HTML source panel. You can often hover over the icon to locate it. 
5. Right-click on the image name, hover over it, or open the image in a new tab to find an expanded file path. The specific filepath will vary by browser and operating system. The image below shows one way to view the expected path in Chrome. For example, on my Windows machine my browser expected to find the file at C://Users/username/AppData/Temp/Images/myimage.jpg. Our goal now is make sure that there's a copy of our image in this location. 
   
   
   
6. Open or download a file syncing tool. For Linux or people comfortable with the command line, I recommend Rsync. Right now I'm using Windows, so I used SyncFolder. 
7. Sync these two folders. The source directory should be Twine/Stories/images/ and the destination directory should be the one from your browser, C://username/AppData/Temp/Images/. If syncing feels too complicated to you, you can also just manually copy and paste your images to the destination directory.
8. Every time you add an image, alter media, change the name of a file, or close/re-open your browser, you may need to Sync that folder again. You can set up your syncing tool to run frequently, or you can manually run the sync job by pressing 'Sync' in your tool.

You might be wondering: Why not just store my images in the destination folder in the first place? That's not not ideal for two reasons. First, these temporary directories are sometimes automatically deleted by your operating system to clear up space. If you only store your images there, you may lose them to deletion! Second, when you finish your story and are ready to publish it using Twine, Twine expects to find those images in a "nearby" folder with relative paths. If you upload your media online or want to send your game to others, you will need to send them the media as well. 

Let me know if this helped you on your Twine journey, or if you have a different trick you used instead. 

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

A blessing for mishmash family

A blessing for mishmash family

Maggie Oates

You are invited to unmute. If the words speak to you or resonate with you, please respond with the words in italics.

There are photos we point to, saying "she looks so much like you!" Twins sharing a secret handshake. A medical history, an inheritance.

Blessed is the family bound by blood. 

There are beacons of food, song, or craft, pulling us together like gravity. 

Blessed is the family bound by senses. 

Ocean Vuong notes, “In Vietnamese, the word for missing someone and remembering them is the same.” When his mother asks him over the phone in Vietnamese: Do you miss me? he flinches, thinking she meant: Do you remember me? 

He is honest, “I miss you more than I remember you.” 

Blessed is the family missed. 

Blessed is the family not remembered. 

There are places where we have precious refrigerator rights. Dinners at another’s table. A candy from a neighbor. The child that always smiles and drools on you on your usual bus route. 

Blessed is the family borrowed.

Blessed is the family stumbled upon.

There are names and relationships lost to time or to violence. African Americans honoring their ancestors face “the brick wall” of documentation, with few written records of Black families before emancipation. 

Blessed is the family unknown, but felt. 

There are reunions, first-unions, births, awkward hugs and long excavations through time and memory. 

Blessed is the family found.

There are lovers of leaving. As HP Rivers says, “leaving family and familiarity, leaving tables where love is not being served.” Those who could not see a light in us, maybe a queerness or a disability. 

Blessed is the family that needs to be grieved.

There are strange and defiant mishmashes. Ones who clung together at first in survival. Then, held each other gently in thriving.

Blessed is the family that is chosen.

There are ones we will never meet, never know, never find, never even consider. The ones seen in church pews or quiet libraries or playgrounds. May we quietly reach to embrace them.

Blessed is the family bound by spirit.

Blessed is our family.

---

Written by Maggie Oates for the February 21, 2021 online Allegheny Unitarian Universalist Church service with Rev. Deryck Tines called "Celebrating Family in Black History Month." 

1. Ocean Vuong's words are from his book, On Earth We're Briefly Gorgeous. You can find it in bookstores and libraries. The full quote is

In Vietnamese, the word for missing someone and remembering them is the same: nhớ. Sometimes, when you ask me over the phone, Có nhớ mẹ không? I flinch, thinking you meant, Do you remember me? I miss you more than I remember you.

2. It is very difficult for African-Americans to research their ancestry before the 1900s because "enslaved African Americans were rarely recorded by name in documents of any kind, making the tracing of their antebellum ancestry nearly impossible." However, sometimes this wall is possible to scale. For resources, see Henry Louis Gates Jr and Meaghan Siekman's "Cheat Sheet for Researching African-American Ancestors" and this list of "African American Resources for Pennsylvania." Locally, there is a chapter of the Afro-American Historical and Genealogical Society.

3. HP Rivers wrote "Blessed Are the Queer."

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

What does freedom feel like? (Guided Meditation)

This 5-minute guided meditation was written for an online service on Liberation at Allegheny Unitarian Universalist Church, to be read by Sue Watts. 

What does freedom feel like?

Have you felt freedom, felt it in your body? Where does freedom lie in your body? Summon a bit of it, if you can, for just for a moment.

If your freedom soars, summon that rising in your chest. If your freedom sits and settles, summon that good weight around your hips. If your freedom dances, summon that electricity on your skin. If your freedom is lightness or floating, banish that lump in your throat and ask your limbs to let go of their tightness.

[Pause, 30 sec]

Have you witnessed the freedom of another? Seen a woman walk from prison? Seen a dog with ears flapping in the wind? Seen a child stick up for another when they know something is Right? Seen a man kiss a man on the street and in the sun?

How does it feel to witness freedom?

[Pause, 30 sec]

Have you witnessed the chains of another? Opened your eyes to pain you couldn’t see before? Lifted the veil on a horror where oppressors had taped it down, refusing to let you see?

How does it feel to witness chains?

[Pause, 30 sec]

Have you ever moved your body toward liberation? Not just lifting a helping hand, but moving your whole being? As Aboriginal activists said in the 70s, “If you have come here to help me, you are wasting your time. But if you have come because your liberation is bound up with mine, then let us work together.”

How does it feel to be bound up with me?

[Pause, 30 sec]

What does freedom feel like? Can you expand this freedom?
Can you move your whole being toward freedom?
Can you move your whole being toward your freedom, and mine?

[Pause, 30 sec]

---

Words by Maggie Oates and Sue Watts. They were written for the voice and spirit of Sue Watts in mind. I am not associated with the following people or organizations, but this meditation was heavily inspired by the work of adrienne maree brown (particularly her book Pleasure Activism) and by all the wisdom and teachings of generative somatics.

The quote “If you have come here to help me, you are wasting your time. But if you have come because your liberation is bound up with mine, then let us work together.” was popularized by Aboriginal activist Lilla Watson. It can be credited to Aboriginal activist groups, Queensland, 1970s.

If you use this work, I strongly suggest you consider a donation to individuals or organizations working at the intersection of liberation and body, such as generative somatics or Black Organizing for Leadership and Dignity (BOLD).


"What does freedom feel like?" by Maggie Oates and Sue Watts is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

List of Machine Learning Projects That Will Inevitably Be Used to Identify Your Nudes

Anus print detection

A mountable toilet system for personalized health monitoring via the analysis of excreta. 6 April 2020. https://www.nature.com/articles/s41551-020-0534-9

Each user of the toilet is identified through their fingerprint and the distinctive features of their anoderm, and the data are securely stored and analysed in an encrypted cloud server.

Areola and Nipple Detection "For Criminal Identification"

https://arxiv.org/pdf/1905.11651.pdf

Log In To Cam Sites with Penis Detection

(This project didn't go anywhere)

https://www.camsoda.com/labs/dickometrics/

Ear print detection

https://dl.acm.org/doi/abs/10.1145/2431211.2431221?

Blurring doesn't always work

https://ui.adsabs.harvard.edu/abs/2019arXiv190210885P/abstract

(Will be updated as more arise.)

Public Comment on AI for the Defense Innovation Board

On March 14, 2019, the Defense Innovation Board held a listening session on Artificial Intelligence Principles at Carnegie Mellon. The DIB is an independent committee made up mainly of tech execs and academics that advises the DOD. I wasn't planning on giving a comment at the session, but I got so angry listening to the pre-written publicity statements read aloud by defense contracting tech companies, that I couldn't stay seated.

Here's my very last-minute statement:

Hello, my name is Maggie Oates. I'm a PhD student here in Societal Computing and I work in Cylab, which is the cybersecurity lab here.

Let me first say that I disagree with the very enterprise and existence of the Defense Innovation Board, as it seems like a tool to lend credibility to the project of advancing military efficiency and further escalating the baseline of 'defense.' Second, I disagree with CMU's continued involvement with the military and am hard-pressed to think of an ethical and responsible use of AI at all [in defense].
That said, I would like to focus on something else today and that is what I view as an externality of the project of AI in the DOD. And that is the growth of civil surveillance, both domestically and abroad. The development of machine learning algorithms relies on massive amounts of data, of course. And while methods are being developed to reduce the amount of data required, or to reduce the amount of labeled data required, these methods often correlate with having the downside of being hard to explain and more difficult to verify, making them an unlikely use in the DOD's context. Beyond that, the project of labeling data often rests on exploitative labor practices. So I stand here to assert that any responsible principles must address the effects that DOD AI will have on surveillance, not only from the state, but also from the tech companies that will be the first line in building that AI. This topic is absolutely not out of scope. Thank you.

You can view the video and comments on the DIB site.