More Unexpected Applications of grep

_Good artists copy, great artists steal._(1)Quentin Tarantino said “I steal from everything. Great artists steal, they don’t do homages.” He stole that from Pablo Picasso, who said “When there’s anything to steal, I steal,” and “Good artists copy, great artists steal.”

In that frame of mind, let’s see a very elegant use of grep that, believe it or not, I learned from seeing it casually used in someone’s blog’s comments. Add this to the grep tricks I showed you several weeks ago.

In Learning Tree’s Linux optimization and troubleshooting course we do a lot of exploring and experimenting with kernel data structures under /sys and /proc for performance tuning. For example, improving disk I/O performance by selecting and tuning the block storage I/O scheduler and tuning virtual memory management. Also, improving network performance by modifying the packet queue lengths and TCP memory management.

We’re especially interested in settings under /proc/sys/net and /proc/sys/vm. I became curious — how do some of the major players set these parameters?

I run a number of sites hosted at GoDaddy, and of course I chose Linux hosting. So, I can connect via SSH as an unprivileged user and look at individual settings with sysctl or simply cat. But what about collecting all the settings under /proc/sys?

Idea #1

With an interactive shell I could do something like this:

$ more $(find /proc/sys/net -type f | sort)

That would give me at least four lines of output per file, each of which is a kernel data structure: a line of 14 colons, the file name, another line of 14 colons, then the file contents. The data structures I’m interested in are all single lines, usually a single number.

$ more $(find /proc/sys/net -type f | sort)
::::::::::::::
/proc/sys/net/core/dev_weight
::::::::::::::
64
::::::::::::::
/proc/sys/net/core/message_burst
::::::::::::::
10
--More--(Next file: /proc/sys/net/core/message_cost)

However, that just pages me through the files interactively, and it doesn’t work as wanted through SSH:

$ ssh user@example.com 'more $(find /proc/sys -type f | sort)' > /tmp/godaddy-settings

Idea #2

Maybe I could do this with a recursive copy?

$ mkdir godaddy-proc
$ scp -r user@example.com:/proc/sys godaddy-proc

The result is a little surprising and quite disappointing: I get the tree of subdirectories and files, but the files are all empty.

Inspiration Strikes! (Or is Stolen)

Here is the way to solve the problem:

$ ssh user@example.com 'grep . $( find /proc/sys -type f | sort )' > godaddy-settings

Let’s look at this a piece at a time. First, the command substitution in $(...). That’s a find piped into sort, the result is an alphabetical list of all the files (really, kernel data structures) under /proc/sys.

Second, we’re running things over on example.com having authenticated there as user with an SSH key exchange.

Third, what’s we’re doing as user@example.com is using grep to select all the lines containing “.”, but that regular expression matches any single character. Since grep was called with a long list of file names, each line of output will start with the file name, then “:”, then the matching line.

Finally, all that output goes into a file named godaddy-settings

I took the idea for this from a blog comment where someone had suggested this simpler local version to check some TCP timing issues:

$ grep . /proc/sys/net/ipv4/tcp*

The Result

The result is one line per file, with the file name separated by a colon from the contents:

$ more godaddy-settings
/proc/sys/crypto/fips_enabled:0
/proc/sys/dev/parport/default/spintime:500
/proc/sys/dev/parport/default/timeslice:200
/proc/sys/dev/raid/speed_limit_max:200000
/proc/sys/dev/raid/speed_limit_min:1000
/proc/sys/dev/rtc/max-user-freq:64
/proc/sys/dev/scsi/logging_level:0
[... hundreds more lines follow ...]

That’s it for this time! Meanwhile I will be analyzing these GoDaddy suggestions.

Type to search blog.learningtree.com

Do you mean "" ?

Sorry, no results were found for your query.

Please check your spelling and try your search again.