The Magic of Logparser

I had a slew of IIS FTP logs to dig through for a work project. I needed unique visitor details, and I needed it from a month of logs. Some of these daily logs were in excess of 40mb. I threw together a PowerShell script to do just so, kicked it off, and waited. And waited. I reconfigured the script to be more efficient in its processing, kicked it off again, and waited. And waited. And researched, because this was taking far too long. And – late to the game, I know – I found Microsoft’s Logparser. Logparser is a free tool, and it’s filled with magic. With one simple query it managed to pull the raw data I needed from the log files in *seconds*.

#log files in c:\temp\iis, results to same path

logparser "SELECT DISTINCT c-ip, cs-username INTO C:\temp\iis\results.txt FROM 'C:\temp\iis\*.log'" -i:W3C -o:W3C


In less than a minute I had a text file of unique visits gleaned from 550 MB worth of plaintext log files. From there I could use PowerShell to filter out any IP dupes (some connections do not have user details, thus producing some leftover duplication) and resolve hostnames from IP addresses to create a final masterlist. I’m not sure how Logparser does what it does so quickly.

Post a comment

You may use the following HTML:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">