[cvsnt] "history" command returns -1

Fri Feb 1 14:24:35 GMT 2008

Nicolas Gavard wrote:
> Thanks chuck for the reply,
> 
>> What you'll probably find is that your history file has some minor 
>> corruption.
> 
> Can you explain what kind of corruptions may have my history file?
> Is it easy to fix? And how those corruptions may happen?
> 

The corruption will look like a truncated line.  In a previous thread 
Tony explained how he believes they happen.  We've just come to accept 
that history is "mostly complete," but may be missing some lines.

>> Older versions of cvs would ignore these lines, the latest versions of 
>> cvsNT treat it as an error and return -1.
>> We run a cron job hourly to clean up the history file.  I can give you my 
>> python script if you like.
> 
> Yes. Please.
>  
Script below.  It does a read-only check to see if it needs to do the 
fix because file copying is pretty expensive.  If your repository is 
very active, you run the risk of losing a line or so when it copies from 
the temp file back to the real file.  That will probably affect your 
scheduling decision.

>> The solution that was recommended to me was to turn off the history file 
>> and turn on auditing.
>> That requires a database, and of course the history command no longer 
>> works.  Instead you need to do sql queries against the database.  This 
>> approach may suit you better.  It wasn't the right solution for us.
> 
> I wouldn't turn off history file. My aim is precisely to make it work with a
> build automation tool (luntbuild) in order to improve performance.
> 

The other thing we do is to split off the beginning of the history file 
once a month to make it smaller and faster.  The history file in our 
most active repository runs about 1 Gb most of the time with 3-4 months 
of data.  So we keep the old history files in case we need them, but 
they don't participate in the cvs history command.

> Thanks again,
> 
> NG.
> 
> 
> 
#!python
import os
import string
import stat
import sys
import re
import time
import shutil

logFile = None

#-------------------------------------------------------------------------------------------
# main
#-------------------------------------------------------------------------------------------
def main():

     # 
O4554d7e3|Joe.User|JoeUser.company.com/repos/dir/plot/PrintMana*17|repos/dir/plot/PrintManager|BR_product080903xx|repos/dir/plot/PrintManager|

     historyRe = re.compile ("^[a-zA-Z]+[0-9a-z]+\|.*\|.*\|.*\|.*\|.*\|")

     if len(sys.argv) != 2:
         print ("usage:  python fixhistory.py filename")
         sys.exit (1)

     historyFileName = sys.argv[1]

     # Check for any errors
     historyErrors = 0
     historyFile = open (historyFileName, 'r')

     lineNum = 0

     line = historyFile.readline()
     while 1:
	if not line:
	    break;
         if not historyRe.match (line):
             historyErrors = 1
             break
         line = historyFile.readline()
         lineNum = lineNum + 1

     historyFile.close()

     if historyErrors == 0:
         print "no problems found"
         sys.exit (0)
     else:
         print "** fixing history file **"

     tempFileName = os.path.join (os.path.dirname (historyFileName), 
"history.fix.tmp")
     tempFile = open (tempFileName, 'wb')

     historyFile = open (historyFileName, 'r+b')

     line = historyFile.readline()
     while 1:
	if not line:
	    break;
         if historyRe.match (line):
            tempFile.write (line)
         else:
             print "line " + `lineNum` + " is bad"
             print '    "' + line[:-1] + '"'

         line = historyFile.readline()

     historyFile.close()
     tempFile.close ()

     tryCount = 0
     while tryCount < 10:
         try:
             tryCount = tryCount + 1
             shutil.copy (tempFileName, historyFileName);
             os.remove (tempFileName)
             historyErrors = 0
             break
         except:
             pass

     if historyErrors == 0:
         print "** history file fixed **"
         sys.exit (0)
     else:
         print "** history file NOT fixed **"
         sys.exit (1)

if __name__ == '__main__':
     main()