json.md

JSON

JavaScript Object Notation.

For JSON basics, read Wikipedia.

Useful JSON tools:

• gron
• jq
  • jq tips
    • jq default value
    • jq find field anywhere in struct
• jnv
• jgrep
• Java Libraries
• Conversion
  • XML to JSON
  • JSON to YAML
  • JSON to CSV
• Pretty Print / format JSON
• JSON Linting
• JSON Comments
  • IDEs
• Memes
  • JSON Statham
  • Trump Tariff CSV Imports

gron

tomnomnom/gron

Flattens JSON to be greppable.

gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" |
fgrep "commit.author"

unflatten back to expanded JSON:

gron "https://api.github.com/repos/tomnomnom/gron/commits?per_page=1" |
fgrep "commit.author" |
gron -u

jq

JSON Query filters json inputs - file or stdin.

Pre-compiled C available for all platforms.

Widely used in my scripts in DevOps-Bash-tools repo.

https://jqlang.github.io/jq/tutorial/

https://jqlang.github.io/jq/manual/

jq tips

jq default value

jq returns literal null string for fields that don't exist, this is annoying af in bash scripts where you will often be testing for failing to find something using [ -z "$output" ].

To avoid this and have jq return a default value, blank in this case, instead of a literal null:

jq -r '.non_existent_key // ""' < file.json

jq find field anywhere in struct

Find and output the policy field from anywhere in the JSON. Most of the time you should understand the schema and specify it explicitly.

jq -r '.. | objects | select(has("policy")) | .policy'

• .. recurses
• objects filters to only objects (ignores arrays & scalars)
• select(has("policy")) returns only objects with a policy field
• .policy selects that field out of those subset of objects

jnv

ynqa/jnv

Interactive JSON viewer.

Useful to quickly create jq queries by drilling down interactively.

cat data.json | jnv

or

jnv data.json

Tab completion.

Ctrl-q - copy jq query to clipboard to paste to code.

Ctrl-o - copy the JSON to clipboard.

More Keyboard shortcuts.

jgrep

JSON grep.

ploubser/JSON-Grep

http://jgrep.org/

Java Libraries

• Jackson
• Gson

Conversion

XML to JSON

xml2json

JSON to YAML

yq -P < "$file"

or from DevOps-Bash-tools:

json2yaml.sh "$file"

or

json2yaml.sh < "$file"

This will figure out which of the following you have installed and use the first one it finds in this order:

ruby -r yaml -r json -e 'puts YAML.dump(JSON.parse(STDIN.read))' < "$file"

python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' < "$file"

This script doesn't use these any more:

• yq - there were two different ones found in $PATH that could result in different results:
• Perl JSON::XS
  • sorted the keys, losing the original structure of the file where the variables were at the top for human readability
  • found when converting a json to yaml that it converted this:

"ssh_pty": true

to this useless thing:

ssh_pty: !!perl/scalar:JSON::PP::Boolean 1

JSON to CSV

json2csv

Pretty Print / format JSON

Perl JSON::XS CPAN module (doesn't sort keys):

json_xs

Python 2.6+ (sorts keys):

python -m json.tool

JSON Linting

From DevOps-Bash-tools, recursively find and lint all *.json files:

check_json.sh

From DevOps-Python-tools, recursively find and lint all *.json files:

validate_json.py .

I run these automatically in all my GitHub repos via CI/CD.

JSON Comments

JSON does not allow comments like # or // in programming languages.

If you try to add them it will break linting and real-world parsing of the JSON.

JSON comment support was removed intentionally because some bad programmers were using them to contain things like parsing directories, breaking parsing compatibility between systems.

As a workaround, you can create key pairs with unused keys like _comment:

{
  "_comment": "This is a commment about why this JSON content sucks",
  "name": "Hari Sekhon",
  "_comment2": "Another comment must have a unique key"
}

IDEs

• Eclipse JSONTools validation plugin (Help -> MarketPlace)
  • needs file extensions to be .json (not .template from AWS CloudFormation)
• IntelliJ also has JSON error validation but it's not as easy to see underscores vs the big red cross Eclipse puts in the left column

Memes

JSON Statham

Trump Tariff CSV Imports