Quentin Tarantino said “I steal from everything. Great artists steal, they don’t do homages.” He stole that from Pablo Picasso, who said “When there’s anything to steal, I steal,” and “Good artists copy, great artists steal.”
In that frame of mind, let’s see a very elegant use of grep that, believe it or not, I learned from seeing it casually used in someone’s blog’s comments. Add this to the grep tricks I showed you several weeks ago.
In Learning Tree’s Linux optimization and troubleshooting course we do a lot of exploring and experimenting with kernel data structures under /sys and /proc for performance tuning. For example, improving disk I/O performance by selecting and tuning the block storage I/O scheduler and tuning virtual memory management. Also, improving network performance by modifying the packet queue lengths and TCP memory management.
We’re especially interested in settings under /proc/sys/net and /proc/sys/vm. I became curious — how do some of the major players set these parameters?
I run a number of sites hosted at GoDaddy, and of course I chose Linux hosting. So, I can connect via SSH as an unprivileged user and look at individual settings with sysctl or simply cat. But what about collecting all the settings under /proc/sys?
With an interactive shell I could do something like this:
$ more $(find /proc/sys/net -type f | sort)
That would give me at least four lines of output per file, each of which is a kernel data structure: a line of 14 colons, the file name, another line of 14 colons, then the file contents. The data structures I’m interested in are all single lines, usually a single number.
$ more $(find /proc/sys/net -type f | sort) :::::::::::::: /proc/sys/net/core/dev_weight :::::::::::::: 64 :::::::::::::: /proc/sys/net/core/message_burst :::::::::::::: 10 --More--(Next file: /proc/sys/net/core/message_cost)
However, that just pages me through the files interactively, and it doesn’t work as wanted through SSH:
$ ssh user@example.com 'more $(find /proc/sys -type f | sort)' > /tmp/godaddy-settings
Maybe I could do this with a recursive copy?
$ mkdir godaddy-proc $ scp -r user@example.com:/proc/sys godaddy-proc
The result is a little surprising and quite disappointing: I get the tree of subdirectories and files, but the files are all empty.
Here is the way to solve the problem:
$ ssh user@example.com 'grep . $( find /proc/sys -type f | sort )' > godaddy-settings
Let’s look at this a piece at a time. First, the command substitution in $(...). That’s a find piped into sort, the result is an alphabetical list of all the files (really, kernel data structures) under /proc/sys.
Second, we’re running things over on example.com having authenticated there as user with an SSH key exchange.
Third, what’s we’re doing as user@example.com is using grep to select all the lines containing “.”, but that regular expression matches any single character. Since grep was called with a long list of file names, each line of output will start with the file name, then “:”, then the matching line.
Finally, all that output goes into a file named godaddy-settings
I took the idea for this from a blog comment where someone had suggested this simpler local version to check some TCP timing issues:
$ grep . /proc/sys/net/ipv4/tcp*
The result is one line per file, with the file name separated by a colon from the contents:
$ more godaddy-settings /proc/sys/crypto/fips_enabled:0 /proc/sys/dev/parport/default/spintime:500 /proc/sys/dev/parport/default/timeslice:200 /proc/sys/dev/raid/speed_limit_max:200000 /proc/sys/dev/raid/speed_limit_min:1000 /proc/sys/dev/rtc/max-user-freq:64 /proc/sys/dev/scsi/logging_level:0 [... hundreds more lines follow ...]
That’s it for this time! Meanwhile I will be analyzing these GoDaddy suggestions.
