Daily history files for Bash

Here’s a little tip for you command-line junkies (like me!). HISTSIZE never gives me enough history, so I decided to keep everything instead. Here’s how…

Edit your ~/.bash_profile (or whatever startup script you use) and add this to it:

export HISTFILE=~/.history/`date +%Y%m%d`.hist
export HISTSIZE=100000

And you would also create a ~/.history directory to hold your history files. This will create one file per day of all your bash sessions.

If you prefer to keep one per week, use this assignment instead (files are named as year-week.hist using Monday as the first day of the week. If you prefer to start your week of history on Sunday, use ‘%U’ instead of ‘%W’ below):

export HISTFILE=~/.history/`date +%Y-%W`.hist

So there you go— note that if you have a session that spans two days, the history will continue to be logged to the original date of the bash session. This is convenient, since it insures that a session is never split up between two history files.

The only drawback to this technique is that when you login at the start of the day, you have no history from the prior day to recall commands used then. If you want, you can draw the last “X” entries from the previous day’s history, rolling it forward if you will. If you’re using this as a way to log your activity for the day, it pollutes your logs a little.

Here’s what you need to prepopulate new history files with 50 commands from the last session:

export HISTFILE=~/.history/`date +%Y%m%d`.hist
if [[ ! -e $HISTFILE ]]; then
    LASTHIST=~/.history/`ls -tr ~/.history/ | tail -1`
    if [[ -e $LASTHIST ]]; then
        tail -50 $LASTHIST > $HISTFILE
        # Write a divider to identify where the prior day's session history ends
        echo "##########################################################" >> $HISTFILE
export HISTSIZE=100000

Then you can use tools like grep to help find commands within the last month:

grep sudo ~/.history/200605*

Or the last year:

grep sudo ~/.history/2006*

Or the last few days:

grep sudo ~/.history/2006051[6-8]*


TrackBack URL for this entry:


Yann said:

what about keeping a smaller HISTSIZE, and using a cron instead to populate .history with backups from .bash_history ?

This way you shouldn't have duplicates in your backup while keeping the usual continuous .history file for your day-to-day history scrubbing.

reeses said:

I needed to be a snoop and log everything a particular user was doing and keep multiple login shell sessions separate. As a result, I added the time of day to $HISTFILE.

Then I wondered why it didn't work. I logged in, I su -'d, and the file never appeared. I could touch the file and create it in ~ from the command line.

Finally, I added the touch ~/blah to the .profile, and received an error message. Changing ~ to the full path was sufficient to solve the problem.

This obviously isn't a general problem (~ is not a traditional home, but more like having ~apache be /var/apache), but because bash appears to fail silently if $HISTFILE is not "valid" for whatever reason, a quick "touch $HISTFILE" added to the .profile should give more helpful diagnostic output.

Anonymous coward said:

This only works if you crack open a fresh terminal everyday right? I've had the same terminal open for about 3 weeks now...

scruzia Author Profile Page said:

I usually have 3 to 5 terminal windows open at once. Does Bash manage to share a history file in that situation? Zsh doesn't (as of v4.2.3 on Mac OS X). So here's what I ended up doing in one of my zsh startup files:

tdc_day=$( tdc_sh | sed 's/_.*//' ) # my date stamper
tty=$( tty | sed 's,/dev/,,' ) # tty always prepends the useless "/dev/".
export HISTFILE=$h/zsh.histories/zh.$tdc_day.$tty
export HISTSIZE=100000

Google for "gigo beh" to see more about my date stamp format.

bronosky.com Author Profile Page said:

Find HISTTIMEFORMAT on http://www.gnu.org/software/bash/manual/bashref.html#Bash-History-Facilities

That var, if set will cause every [new] line in HISTFILE to be prefaced with a line that looks like the result of:
echo "#$(date +%s)"
Regardless of the format of HISTTIMEFORMAT. The bash builtin history command will use that format to display the date for each command. No whitespace is added, so you must put it in your format. I use:
export HISTSIZE=10000
shopt -s histappend

I use a cron job to sweep old commands into an archive. I have a similar approach to vim. I can tell you every command I've executed and every file I've saved since the dawn of time.


This article was published on May 19, 2006 2:20 PM.

The article previously posted was Tunneling SubEthaEdit.

The next article is Big Spam and SpamAssassin.

Many more can be found on the home page or by looking through the archives.

Powered by Movable Type