More Unexpected Applications of grep

_Good artists copy, great artists steal._(1)Quentin Tarantino said “I steal from everything. Great artists steal, they don’t do homages.” He stole that from Pablo Picasso, who said “When there’s anything to steal, I steal,” and “Good artists copy, great artists steal.”

In that frame of mind, let’s see a very elegant use of grep that, believe it or not, I learned from seeing it casually used in someone’s blog’s comments. Add this to the grep tricks I showed you several weeks ago.

In Learning Tree’s Linux optimization and troubleshooting course we do a lot of exploring and experimenting with kernel data structures under /sys and /proc for performance tuning. For example, improving disk I/O performance by selecting and tuning the block storage I/O scheduler and tuning virtual memory management. Also, improving network performance by modifying the packet queue lengths and TCP memory management.

We’re especially interested in settings under /proc/sys/net and /proc/sys/vm. I became curious — how do some of the major players set these parameters?

I run a number of sites hosted at GoDaddy, and of course I chose Linux hosting. So, I can connect via SSH as an unprivileged user and look at individual settings with sysctl or simply cat. But what about collecting all the settings under /proc/sys?

Idea #1

With an interactive shell I could do something like this:

$ more $(find /proc/sys/net -type f | sort)

That would give me at least four lines of output per file, each of which is a kernel data structure: a line of 14 colons, the file name, another line of 14 colons, then the file contents. The data structures I’m interested in are all single lines, usually a single number.

$ more $(find /proc/sys/net -type f | sort)
--More--(Next file: /proc/sys/net/core/message_cost)

However, that just pages me through the files interactively, and it doesn’t work as wanted through SSH:

$ ssh 'more $(find /proc/sys -type f | sort)' > /tmp/godaddy-settings

Idea #2

Maybe I could do this with a recursive copy?

$ mkdir godaddy-proc
$ scp -r godaddy-proc

The result is a little surprising and quite disappointing: I get the tree of subdirectories and files, but the files are all empty.

Inspiration Strikes! (Or is Stolen)

Here is the way to solve the problem:

$ ssh 'grep . $( find /proc/sys -type f | sort )' > godaddy-settings

Let’s look at this a piece at a time. First, the command substitution in $(...). That’s a find piped into sort, the result is an alphabetical list of all the files (really, kernel data structures) under /proc/sys.

Second, we’re running things over on having authenticated there as user with an SSH key exchange.

Third, what’s we’re doing as is using grep to select all the lines containing “.”, but that regular expression matches any single character. Since grep was called with a long list of file names, each line of output will start with the file name, then “:”, then the matching line.

Finally, all that output goes into a file named godaddy-settings

I took the idea for this from a blog comment where someone had suggested this simpler local version to check some TCP timing issues:

$ grep . /proc/sys/net/ipv4/tcp*

The Result

The result is one line per file, with the file name separated by a colon from the contents:

$ more godaddy-settings
[... hundreds more lines follow ...]

That’s it for this time! Meanwhile I will be analyzing these GoDaddy suggestions.

Type to search

Do you mean "" ?

Sorry, no results were found for your query.

Please check your spelling and try your search again.