Skip to main content

Primary links

  • Home
  • AI
  • Kubernetes
  • Incus
  • Ansible
  • Terraform
  • OpenStack
  • Virtualization
  • Linux
  • SmartHome
  • HowTo

Misc

  • Linux
  • Hardware
  • Programming
    • Graphic
    • HTML / PHP / CMS
    • Java
    • PL / SQL
    • Magento
    • python
    • Asterisk
    • C / C++
    • JavaScript
    • Json
    • Magento
    • OpenOffice / LibreOffice
    • Perl
    • Postfix
    • Regular Expression
    • TLA
    • Tomcat
    • Zenity
    • bash
    • find
    • git
    • ofbiz
    • wildfly
    • xml
  • Databases
  • CLI
  • Multimedia
  • Plugins
  • Windows
  • Contact

Cloud

  • OpenStack
  • cloud-config
  • nextcloud

Virtualization

  • Virtualization
  • Docker
  • KVM
  • Kubernetes
  • LXC
  • LXD
  • QEMU
  • VMware
  • VirtualBox
  • multipass
  • podman
  • vagrant
  • XEN
  • Incus

Network

  • DNS
  • Firewall
  • Linux
  • OpenvSwitch
  • SSL
  • VLAN
  • VPN
  • iPXE
  • namespaces
  • nmcli
  • tcpdump

Storage

  • CEPH
  • DRBD
  • LVM
  • S3
  • ZFS
  • btrfs

Automation / CI/CD

  • Install
  • Ansible
  • GitLab
  • LLM
  • Preseed
  • Puppet
  • Terraform
  • Ubuntu autoinstall

Monitoring / Visualisation

  • Grafana
  • Icinga
  • Prometheus
  • Monitoring
  • ELK
  • mermaid
  • json

Install

sudo apt install -y jq

Parse

# select
jq 'select(.geo != null)' all.json
wget https://releases.hashicorp.com/index.json -qO- | jq -r 'last(.vagrant.versions[]).builds[] | select(.os == "debian" and .arch == "x86_64").url'

# contains
echo "$json" | jq -c '.[] | select(.genre | contains("foo"))'

# select multiple conditions
jq -r 'select((.vendor == null) or (.vendor | contains("AVAGO") | not)).product'

# select value
jq -r '.[] | select(.protocol_port == 443) | .id'

# read two values and search from file
jq -r '.GeneralInfo.UUID + " " + (.NetworkEntry[] | select(.Notes | contains("-mgmt")).Entry)' *.json

# not contain
jq -r '.[] | select(.status | contains("CREATE_COMPLETE") | not).uuid'

# search
jq '[ .devices | select(.key | startswith("dimmer")) | .value = .value.state ] | from_entries' filename.json
jq -r '.[] | select(."Fixed IP Addresses"[].ip_address | startswith("10.11")).ID'

# pretty print
cat /tmp/in.json | python -m json.tool

# JSON Validator
https://jsonformatter.curiousconcept.com/

# parse key with spaces
cat x.json | jq '."General Info"."Model No."'

# marge JSON files
jq -s '.' /tmp/*.json > out.json
jq 'select(.host.nic1 != null)' *.json | jq -s '.'

# filter not empty
jq -r 'select(length > 0) | .Devices[].ModelNumber'

# filter null value
jq -r '.properties.sysadmin | select (.!=null) 

# convert value to lowercase
jq -r '.foo | ascii_downcase

# output without quotes
cat file.json | jq -r '.name'

# key containing blank
storcli /c0/v0 show J | jq -r '.Controllers[0]."Response Data"."Virtual Drives"[0].State'

# show multiple values
jq -r '.[].Name, .[].HostConfig.RestartPolicy.Name'
jq -r '.[] | (.Name + " " + .HostConfig.RestartPolicy.Name)'

# show multiple values in multiple lines with additional text
jq -r '"my_id: " + .id, "my_secret: " + .secret'

# join lines
jq -r '.security_group_ids | join(",")')

# delete empty values
jq 'del(.[][] | nulls)'
jq 'del(.[][] | select(. == ""))'
del(.[][] | select(. == "" or .BAR==0))

# split array to line
jq -c '.[]'

# split by
jq -r #'.[]."Fixed IP Addresses" | split(",") | first | split("=") | nth(1)'


# replace null with 0
(.time.total // 0)

# base64 (jq to bash)
echo "${sample}" | jq -r '.[] | @base64'
echo ${row} | base64 --decode | jq -r ${1}

processing Icinga warning Json list

curl -s -u "foo:bar" -k "https://monitoring.example.com/icinga/cgi-bin/status.cgi?servicestatustypes=20&noheader=1&servicestatustypes=29&sorttype=2&sortoption=3&scroll=963&jsonoutput"  | \
  jq --raw-output ' .status.service_status[] | select(.host_name | contains("dev")) | select(.status_information | contains("Apache", "Java")) | .host_name'

Marge files
https://stackoverflow.com/questions/49037956/how-to-merge-arrays-from-two-files-into-one-array-with-jq/

jq -s . /tmp/x.json /tmp/y.json
jq -s '[ .[0] + .[1] | group_by(.name)[] | select(length > 1) | add ]' registration.json useredits.json

Get first / last / Nth element from list

jq -r 'first(.security_group_ids[])'
jq -r 'last(.security_group_ids[])'

( nth(1; .[] | select(.a == "x")) )

# search for dublettes
jq -r .Name *.json | sort | uniq -c | grep -v "      1"

# get array length / size
jq length

# get keys
https://stackoverflow.com/questions/34226370/jq-print-key-and-value-for-each-entry-in-an-object
lldpctl  -f json | jq -r '.lldp.interface[] | keys[] as $k | "\($k) \(.[$k] | .port.id.value)"'
lldpctl  -f json | jq -r '.lldp.interface[] | with_entries(.value |= .port.id.value)'
lldpctl  -f json | jq -r ".lldp.interface[].${NIC} | select(.chassis != null) .chassis | keys[]"
lldpctl  -f json | jq -r ".lldp.interface[].${NIC} | select(.chassis != null) .port.id.value"

Convert csv to json

jq --slurp --raw-input --raw-output \
  'split("\n") | .[1:] | map(split(",")) |
      map({
           "id": .[0],
           "ascension": .[1],
           "declination": .[2],
           "ultraviolet": .[3]})' \
  input.csv > output.json

# add value
jq --arg project_name ${PROJECT_NAME} --arg domain_name ${DOMAIN_NAME} '. + {"Project": $project_name, "Domain": $domain_name}'

# map
jq -Rs 'split("\n") | .[0:] | map(split(" ")) | map({ ID: .[0], Tanent: .[1] }) ' /tmp/server_project.csv
jq -s '[ .[0] + .[1] | add ]' /tmp/server_list.json /tmp/server_project.json

# convert to array
jq --slurp .

# get smallest object
lsblk --include 8 -J | jq -r '.blockdevices | sort_by(.size) | last | .name'

# get first and last element from array
.foo[].bar | first, last

# output as array
| [.address, .host_name] '

YAML parser similar to jq
https://kislyuk.github.io/yq/

Links
https://shapeshed.com/jq-json/
https://stedolan.github.io/jq/manual/
https://jqplay.org/
http://www.compciv.org/recipes/cli/jq-for-parsing-json/
https://github.com/stedolan/jq/issues/82
https://stedolan.github.io/jq/manual/
https://stackoverflow.com/questions/26195214/how-to-nicely-remove-empty-array-in-jq