EQMac.com

Everquest for the Mac discussion
It is currently Tue May 21, 2013 8:47 am

All times are UTC - 10 hours




Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
Author Message
 Post subject: Parser
PostPosted: Thu Jan 20, 2011 2:11 pm 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
EQMac_LogAnalyzer.pl

This is a script I've been working on here and there for the last year or so. It's written in Perl and reads and analyzes your log file. It's reasonable good at summarizing melee combat and tracking spells cast. You should be comfortable using Terminal to use it, since there is no real GUI, and only one forum post for a manual.


How to Install:

1. Download Parser

2. Open Terminal

3. Extract the files from where ever you saved the download, to where ever you want to run it:
Code:
tar -zxvf ~/Downloads/Parser.tgz /Applications/EverQuest/
Three files will be created:
EQMac_LogAnalyzer.pl -- This is the main program.
EQLogParser.pm -- This handle categorizing each line. Adapted from an EQLive CPAN module.
spell_table.pl -- A big database with information on almost every the spells in the game (Thanks Dwin).


How to Run:

1. Open Terminal

2. Go to the directory where the script is installed:
Code:
cd /Applications/EverQuest/

3. Run the script on the file you want to parse. It will print out information about each fight it's able to identify with who did melee damage and how much. Non-melee damage is included but only for yourself and not others:
Code:
./EQMac_LogAnalyzer.pl game/eqlog_CHARNAME_52.txt | less


If you want to save everything it prints to a file named parser_results.txt run:
Code:
./EQMac_LogAnalyzer.pl game/eqlog_CHARNAME_52.txt > parser_results.txt


Now if you want to limit it to the last day (the script recognizes any of these 1w = 7d, 1d = 24h, 1h = 60m -- dont put a space between the number and units):
Code:
./EQMac_LogAnalyzer.pl -most_recent 24h game/eqlog_CHARNAME_52.txt | less


There is a filter to summarize only certain fights. For example if you only want to report fights against mobs with Zek in their name from the last half hour:
Code:
./EQMac_LogAnalyzer.pl -most_recent 30m game/eqlog_CHARNAME_52.txt -filter Zek


Or you can summarize your whole session, and not just individual fights:
Code:
./EQMac_LogAnalyzer.pl -most_recent 6h game/eqlog_CHARNAME_52.txt -stats | less


You could even summarize other players and mobs (or use "-stats all" to get summaries for everyone):
Code:
./EQMac_LogAnalyzer.pl -most_recent 1w game/eqlog_CHARNAME_52.txt -stats NAME | less


For reference, I get about 0.5MB/sec or 8000 lines/sec performance. If you want it to run in seconds instead of minutes (hours if you have a GB+ log file) you could do something like this to look at only the last 25000 lines:
Code:
tail -25000 game/eqlog_CHARNAME_52.txt | ./EQMac_LogAnalyzer.pl - -stats all



But I already use a parser:

Now most of this can be done better by Era appropriate Windows/EQLive parsers. However they can't do this:
Code:
./EQMac_LogAnalyzer.pl -monitor game/eqlog_CHARNAME_52.txt
The program starts but instead of reading all the old lines, it waits for new lines to be added. In this mode you get a realtime fight summary at then end of each fight. Also, it tracks all the spells you cast and reminds you when they are going to fade. If you cast single buffs or short duration debuffs, this can be very helpful.


Todo list:

1.The spell timer currently doesn't handle buff extension AA and focus. The focus items do show up in the log file, but AA would have to provided as an option. I could use some log file snippets with popular focus items to test.

2. It would be nice to estimate spell damage based on emotes, to guess what damage other casters are doing.

3. DOT damage could be estimated based on cast time, fight duration and spell database.

4. Currently there is no real mitigation analysis. A lot of the information is there, but hasn't been extracted and summarized.

5. Crit rates aren't tracked.

6. The combat parser gets confused with charmed mobs fight another mob, then suddenly turn and attack you. The log file doesn't really show which is the pet.

7. Some of the reports it dumps are ugly. I had planned to dump HTML or something that would look nicer. I even thought about writing a GUI on top, but I'm not much of a GUI programmer.


Disclaimer:

This software is freely available to copy and use and redistribute. If you find a bug or make an improvement, I would appreciate if you would post back here or send me a PM so others can benefit.


Let the DPS wars begin!

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Last edited by Zenum on Fri Jan 21, 2011 12:31 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Jan 20, 2011 2:19 pm 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
reserved for future use ~

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Jan 20, 2011 3:08 pm 
Offline
Officer: Temerity
User avatar

Joined: Thu Apr 01, 2010 8:08 pm
Posts: 510
Thanks Zegnome! Can't wait to play around with this :D

_________________
Pithy Deathwish, manic mechanic of <Temerity>

Refuse, rubbish, filth, slime, muck and other alt-putrescence:
[ Drat | Dart | Zounds | Slouch | Pouch ]


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Jan 20, 2011 4:13 pm 
Offline
Member

Joined: Mon Jan 05, 2009 11:32 pm
Posts: 569
Thanks Zenum - having a real EQMac parser is going to be great!

_________________
Ryoku Wing, Swift Tail of Temerity

Soft-skinned alt scum:
[ Ravenwing | Syovan | Prudence | Auvry | Sycorax ]


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Fri Jan 21, 2011 10:05 am 
Offline
Member

Joined: Fri Jul 11, 2008 2:37 pm
Posts: 333
Location: TX, US
Nice work, and nice to see another Perl hacker around here :)

_________________
Image


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Fri Jan 21, 2011 10:49 am 
Offline
Member
User avatar

Joined: Fri Feb 25, 2005 11:16 am
Posts: 799
Location: Chicago
This is cool. Great contribution. Thank you!

_________________
Elder Judah, Harvester of Tares Lichen

V


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Fri Jan 21, 2011 12:21 pm 
Offline
Member

Joined: Fri Mar 27, 2009 4:21 am
Posts: 447
I got a chance to play with this some today. I really appreciate you doing this and posting it. It helps me on deciding some weapons choices; taking it from theoretical to applied. I could always do it using windows apps but this is just better. I see you have plans eventually for some changes and I'll look forward to seeing the program evolve.

_________________
Daro - 65 Monk ** 4/30/2011
Lots of other characters


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Mon Jan 24, 2011 6:38 am 
Offline
Member

Joined: Thu Jul 03, 2008 9:01 am
Posts: 911
Wonder what it does about enchanter pets....

Too bad we can't name the pet after we charm it.

_________________
Sinnder, Enchanter of Destiny
Joshore, Druid of Destiny


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Mon Jan 24, 2011 7:06 am 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
Parsing charmed pets should work fine if you have a real tank, and don't let it actually get close enough to hit anyone on charm breaks. But if the logfile only shows one mob against another mob, especially if the names match, who knows what you'll get. It might help parsing if you pulled with a ceramic band to show which mob you are doing non-melee damage to. That will be the target, and any other NPC that fight against the target will be assumed to be charmed pets.

I forgot to mention it before, but if you are running in monitor mode while playing the game, it helps to do '/who' so the script can learn all the player names in your group / raid. It helps the program resolve the target if you are not the first in combat.

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Jun 30, 2011 12:55 pm 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
I just updated the EQMac_LogAnalyzer program with a few small enhancements. When using -most_recent (or -last) to read only the end of a very large file, it searches backwards from the end instead of forwards from the beginning to find the starting point to parse at. Also I improved the combat stats summary you get from -stats. Also a "-stats players" will summarize all player character results (but not pets for now).

All the details in the first post.

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Fri Dec 09, 2011 7:36 pm 
Offline
Member

Joined: Mon Aug 30, 2010 3:41 am
Posts: 417
Has anyone tried running this with the new client? I tried running the commands in Terminal Zenum had and I couldn't get it to work. I even tried switching it around to reflect where I thought it was downloaded.

_________________
Image
Image


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Fri Dec 09, 2011 11:17 pm 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
Works for me:

Code:
EQMac_LogAnalyzer.pl -monitor Library/Application\ Support/EverQuest/PlayerLogs/eqlog_CHARNAME_52.txt


The new log files have blank lines that cause a harmless error message about timestamp not matching. I uploaded a fix.

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Feb 28, 2013 6:28 am 
Offline
Member

Joined: Wed Mar 17, 2010 9:39 am
Posts: 72
I keep getting the following error:

Code:
player_table.pl did not return a true value at ./EQMac_LogAnalyzer.pl line 168.


Can anyone help my code-addled brain?

_________________
Image


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Feb 28, 2013 7:50 am 
Offline
Member

Joined: Fri Apr 24, 2009 6:24 am
Posts: 109
You should try renaming the file and let the script recreate it.

_________________
Zenum <Dark Conquest>-- 65 Gnome Enchanter

EqChar -- Save your character's profile in a text file.
Zenam's Item List.xls -- (12.1 MB) Pretty much every player usable item on Al'Kabor ranked in a spreadsheet.
EQMac_LogAnalyzer.pl -- (0.31 MB) Script to parse log files for melee DPS plus realtime automatic spell timers! (Forum post)


Top
 Profile  
 
 Post subject: Re: Parser
PostPosted: Thu Feb 28, 2013 9:21 am 
Offline
Member

Joined: Wed Mar 17, 2010 9:39 am
Posts: 72
Now I get:

Code:
Use of uninitialized value in lc at ./EQMac_LogAnalyzer.pl line 463.
Use of uninitialized value in lc at ./EQMac_LogAnalyzer.pl line 463.


And in the parser_results.txt:

Code:
ERROR: date parse failed--{\rtf1\ansi\ansicpg1252\cocoartf1187\cocoasubrtf340
ERROR: date parse failed--{\fonttbl\f0\fswiss\fcharset0 Helvetica;}
ERROR: date parse failed--{\colortbl;\red255\green255\blue255;}
ERROR: date parse failed--\margl1440\margr1440\vieww10800\viewh8400\viewkind0
ERROR: date parse failed--\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\pardirnatural
ERROR: date parse failed--\f0\fs24 \cf0 [Thu Feb 28 01:51:12 2013] Logging to 'eqlog.txt' is now *ON*.\


I need remedial lessons I think!

_________________
Image


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC - 10 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group