Selecting the related endpoints from a GitHub API response using jq's with_entries

| 2 min read

I often find myself searching for the related API endpoints for any given chunk of data returned from a call to the GitHub REST API. Let's take an issue as an example, one related to SAP's Open Documentation Initiative:

Feedback for "Data Lake API": https://github.com/SAP-docs/sap-datasphere/issues/13

Taking the basic information available for this issue via one of the endpoints in the Issues API, we get an object returned with properties that we can list like this:

gh api \
--cache 1h \
repos/SAP-docs/sap-datasphere/issues/13 \
| jq keys

There are quite a few properties, many of them ending _url:

[
"active_lock_reason",
"assignee",
"assignees",
"author_association",
"body",
"closed_at",
"closed_by",
"comments",
"comments_url",
"created_at",
"events_url",
"html_url",
"id",
"labels",
"labels_url",
"locked",
"milestone",
"node_id",
"number",
"performed_via_github_app",
"reactions",
"repository_url",
"state",
"state_reason",
"timeline_url",
"title",
"updated_at",
"url",
"user"
]

A simple with_entries, which is actually just a combination of its two sibling functions: to_entries | map(foo) | from_entries, does the trick:

gh api \
--cache 1h \
repos/SAP-docs/sap-datasphere/issues/13 \
| jq 'with_entries(select(.key | endswith("_url")))'

This gives:

{
"repository_url": "https://api.github.com/repos/SAP-docs/sap-datasphere",
"labels_url": "https://api.github.com/repos/SAP-docs/sap-datasphere/issues/13/labels{/name}",
"comments_url": "https://api.github.com/repos/SAP-docs/sap-datasphere/issues/13/comments",
"events_url": "https://api.github.com/repos/SAP-docs/sap-datasphere/issues/13/events",
"html_url": "https://github.com/SAP-docs/sap-datasphere/issues/13",
"timeline_url": "https://api.github.com/repos/SAP-docs/sap-datasphere/issues/13/timeline"
}

What to_entries, from_entries and with_entries gives us is a way to process properties the names of which are unknown to us until execution time. Each property is normalised into a static structure with well-known property names. Here's an example:

jq -n '{question: "Life", answer: "Forty Two"} | to_entries'

This emits a stable, predictable structure:

[
{
"key": "question",
"value": "Life"
},
{
"key": "answer",
"value": "Forty Two"
}
]

And from_entries reverses the conversion that to_entries performs. For example:

jq -n '
{question: "Life", answer: "Forty Two"}
| to_entries
| map(.value |= ascii_upcase)
| from_entries
'

This emits:

{
"question": "LIFE",
"answer": "FORTY TWO"
}

So this can be replaced simply with:

jq -n '
{question: "Life", answer: "Forty Two"}
| with_entries(.value |= ascii_upcase)
'

This has the same effect (because it's just syntactic sugar for the version with to_entries and from_entries):

{
"question": "LIFE",
"answer": "FORTY TWO"
}

I'm going to try and write more of these short "snippet" posts, to break the cycle of only writing longer, more detailed and therefore less frequent ones. Let's see how it goes.