Read the last line of a text file

Discussion:

(too old to reply)

Jon Paskett

2005-12-08 21:19:46 UTC

I meed to read the last line of a log file and write the result to a text
file. Currently, I am reading every line into an array and it's not too
efficient. How do I just read the last line when I opentextfle?

TIA

tlavedas@hotmail_DOT_com

2005-12-08 22:11:18 UTC

Permalink

This approach uses little code, but is not necessarily the best way to
do it ...

' ----------------------------- Start -------------------------
sFN = WSH.SCRIPTNAME
wsh.echo GetLastLine(sFN)

Function GetLastLine(sFileName)
Const ForReading = 1
With CreateObject("Scripting.FileSystemObject")
With .OpenTextFile(sFileName, ForReading)
GetLastLine = Split(.ReadAll, vbNewline)(.line - 1)
End with
End with
End Function
' ------------------------------ End --------------------------

It will work fine for files up to a few MB, but is certainly not a good
idea for truly BIG files.

AFAIK, the file must be read (in VBS) to get to the end. If you know
the absolute maximum size of the last line, then a
Skip(fso.getfile(sfn).size - len_last_line) approach can me used to get
near the end and then a Split(readAll) approach like above can be
applied.

Tom Lavedas
============
http://my.fcc.net/~tglbatch/wshindex.html (new URL - old content)

Post by Jon Paskett
I meed to read the last line of a log file and write the result to a text
file. Currently, I am reading every line into an array and it's not too
efficient. How do I just read the last line when I opentextfle?
TIA

Al Dunbar

2005-12-09 03:48:20 UTC

Permalink

Post by ***@hotmail_DOT_com
This approach uses little code, but is not necessarily the best way to
do it ...
' ----------------------------- Start -------------------------
sFN = WSH.SCRIPTNAME
wsh.echo GetLastLine(sFN)
Function GetLastLine(sFileName)
Const ForReading = 1
With CreateObject("Scripting.FileSystemObject")
With .OpenTextFile(sFileName, ForReading)
GetLastLine = Split(.ReadAll, vbNewline)(.line - 1)
End with
End with
End Function
' ------------------------------ End --------------------------
It will work fine for files up to a few MB, but is certainly not a good
idea for truly BIG files.
AFAIK, the file must be read (in VBS) to get to the end. If you know
the absolute maximum size of the last line, then a
Skip(fso.getfile(sfn).size - len_last_line) approach can me used to get
near the end and then a Split(readAll) approach like above can be
applied.

Alternately, if you use a simple function in place of .readall that does
this:

determine the size of the file
open the file
use .read to read that many bytes
close the file

you will find this a lot more efficient than .readall, especially for larger
files.

/Al

Christoph Basedau

2005-12-10 02:01:40 UTC

Permalink

For truly big files (>100mb) it is afaics worth saving memory and time
by not allocating string-buffer, but rather opening the file twice.
Once for getting the line-count and a second time for skipping all lines
but the last.

msgbox getlastline("C:\test.log")

function getlastline (ascfile)

set fs = createobject("scripting.filesystemobject")
set file = fs.GetFile(ascfile)
set stream = file.OpenAsTextStream
stream.skip file.size
line = stream.line
stream.close
set stream = file.OpenAsTextStream

for i=1 to line-2
stream.skipline
next
'# fail-safe for CRLF-CRLF EOF
getlastline = stream.readline
'# if the last line is NOT a blank line
if not stream.atendofstream then
getlastline = stream.readline
end if
end function

it's more vb-code but definitely *much* faster for big text files (>100mb)
then using ReadAll+Split or reading it line by line.
For very big files Wscript.exe may even terminate "abnormally"
(VC++ Runtime Error) when trying to ReadAll text into one string-var.

--
Gruesse, Christoph

Rio Riay Riayo - Gordon Sumner, 1979

ekkehard.horner

2005-12-10 21:30:01 UTC

Permalink

[...]
Hi Jon, Tom, Al, Christian,

thanks for the interesting problem/ideas/code. Thanks to you I
spend a nice saturday afternoon.

If you just want the task done, I think Christians's proposal
is the best for the majority of cases (file sizes).

If you want to fine tune your solution to a specific file
size or have slighty different needs (e.g. you want to get
the last n lines from the file) here is some code to
experiment with:

' ############################################################################
''= tests various methods of getting last line(s) of text file
' ############################################################################

' choose size of file
Const cnMB = 5 ' file size (approx)
Const cnLenL = 80 ' average line length
Const cnLstL = 0 ' lines to get (approx) (UBound!)
Const cnLstTL = 2 ' TL: real last line

Dim sFSpec : sFSpec = ".\tailtest.txt"
Dim oFS : Set oFS = CreateObject( "Scripting.FileSystemObject" )
Dim dicMethod : Set dicMethod = CreateObject( "Scripting.Dictionary" )
Dim sKey
Dim fncPtr
Dim aLines

' delete sFSpec if you changed cnMB
If Not oFS.FileExists( sFSpec ) Then
dicMethod.Add "creaData", Array( sFSpec, cnMB, cnLenL, oFS )
End If

' disable tests unsuitable for chosen cnMB by commenting out
dicMethod.Add "testLP" , Array( sFSpec, oFS )
dicMethod.Add "testTL" , Array( sFSpec, cnLstTL )
dicMethod.Add "testCB" , Array( sFSpec )
dicMethod.Add "testEH" , Array( sFSpec, cnLenL, cnLstL, oFS )

WScript.Echo "file size (approx): " & cnMB

For Each sKey In dicMethod
WScript.Echo " ----- " + sKey
Set fncPtr = GetRef( sKey )
Dim dtStart : dtStart = Now
Dim dtEnd
aLines = fncPtr( dicMethod( sKey ) )
dtEnd = Now - dtStart
WScript.Echo " " + Join( aLines, vbCrLf + " " ) + ""
WScript.Echo Space( 7 ) + CStr( CDate( dtEnd ) ) + " for " + sKey
Next

' ============================================================================
''= generates 'random' integer between nFrom and (nTo - 1)
' ============================================================================

Function IRandR( ByVal nFrom, ByVal nTo )
IRandR = nFrom + Int( Rnd * (nTo - nFrom) )
End Function

' ============================================================================
''= creates text file for testing
' ============================================================================

Function creaData( aParms )
Dim sFSpec : sFSpec = aParms( 0 )
Dim nBytes : nBytes = aParms( 1 ) * 1000000
Dim nLenL : nLenL = aParms( 2 )
Dim oFS : Set oFS = aParms( 3 )
Dim oTS : Set oTS = oFS.CreateTextFile( sFSpec )
Dim nLines : nLines = nBytes \ nLenL
Dim nFrom : nFrom = nLenL - ( nLenL \ 5 )
Dim nTo : nTo = nLenL + ( nLenL \ 5 )
Dim nIdx
Dim sTmp

For nIdx = 0 To nLines
oTS.WriteLine nIdx & " " + String( IRandR( nFrom, nTo ), "X" )
Next
sTmp = nIdx & " This is the end * My only friend, the end"
oTS.WriteLine sTmp
oTS.Close

creaData = Array( sTmp )
End Function

' ============================================================================
''= gets last line via loop
' ============================================================================

Function testLP( aParms )
testLP = Array( GetLastLine_LP( aParms( 0 ), aParms( 1 ) ) )
End Function

Function GetLastLine_LP( sFileName, oFS )
Dim oTS : Set oTS = oFS.OpenTextFile( sFileName )
Dim sRVal

While Not oTS.AtEndOfStream
sRVal = oTS.ReadLine
Wend
GetLastLine_LP = sRVal
End Function

' ============================================================================
''= gets last line via ReadAll (Tom Lavedas)
' ============================================================================

Function testTL( aParms )
testTL = Array( GetLastLine_TL( aParms( 0 ), aParms( 1 ) ) )
End Function

Function GetLastLine_TL( sFileName, nLstTL )
Const ForReading = 1
With CreateObject( "Scripting.FileSystemObject" )
With .OpenTextFile( sFileName, ForReading )
GetLastLine_TL = Split( .ReadAll, vbNewline )( .line - nLstTL )
End With
End With
End Function

' ============================================================================
''= gets last line via SkipLine (Christoph Basedau)
' ============================================================================

Function testCB( aParms )
testCB = Array( GetLastLine_CB( aParms( 0 ) ) )
End Function

Function getlastline_CB (ascfile)
Dim fs, file, stream, Line, i
set fs = createobject("scripting.filesystemobject")
set file = fs.GetFile(ascfile)
set stream = file.OpenAsTextStream
stream.skip file.size
line = stream.line
stream.close
set stream = file.OpenAsTextStream

for i=1 to line-2
stream.skipline
next
'# fail-safe for CRLF-CRLF EOF
getlastline_CB = stream.readline
'# if the last line is NOT a blank line
if not stream.atendofstream then
getlastline_CB = stream.readline
end if
end function

' ============================================================================
''= gets last line via Skip & last block read (Ekkehard Horner)
' ============================================================================

Function testEH( aParms )
testEH = getLastLine_EH( aParms( 0 ), aParms( 1 ), aParms( 2 ), aParms( 3 ) )
End Function

Function getLastLine_EH( sFSpec, cnLenL, cnLstL, oFS )
Dim oFile : Set oFile = oFS.GetFile( sFSpec )
Dim oTS : Set oTS = oFile.OpenAsTextStream
ReDim aRVal( cnLstL )
Dim nFSize : nFSize = oFile.Size
Dim nBSize : nBSize = cnLenL * cnLstL + cnLenL + cnLenL
Dim nSSize : nSSize = nFSize - nBSize
Dim sBlk
Dim aBlk
Dim nIdx1
Dim nIdx2

If 0 > nSSize Then
nSSize = 0
End If
oTS.Skip nSSize
sBlk = oTS.ReadAll
sBlk = Replace( sBlk, vbCrLf, vbLf )
If vbLf = Right( sBlk, 1 ) Then
sBlk = Left( sBlk, Len( sBlk ) - 1 )
End If
aBlk = Split( sBlk, vbLf )
nBSize = UBound( aBlk ) - cnLstL
If 0 > nBSize Then
nBSize = 0
End If
nIdx2 = -1
For nIdx1 = nBSize To UBound( aBlk )
nIdx2 = nIdx2 + 1
aRVal( nIdx2 ) = aBlk( nIdx1 )
Next
ReDim Preserve aRVal( nIdx2 )
getLastLine_EH = aRVal
End Function

If you like to pursue the problem of processing large files
fast(er), than you may look at the discussion here:

http://www.visualbasicscript.com/m_23627/tm.htm

Thanks again!