Datamining MS Windows eventlogs converted to xml
Dear, For the longest time a good tool to datamine ms windows eventlogs escaped me. BaseX appears to offer the toolkit which could permit to do so after an affordable conversion to XML. Now i seek to build a set of queries to extract information from multiple converted eventlog files at once. Are there people on this list who have experience or are open to building experience on this topic ? Br, Joris
Hi Joris, Have you already exported the MS windows events to XML, and are you now trying to extract specific information from that files? Best, Christian On Wed, Apr 7, 2021 at 2:13 PM Joris Lambrecht <commandline@protonmail.com> wrote:
Dear,
For the longest time a good tool to datamine ms windows eventlogs escaped me.
BaseX appears to offer the toolkit which could permit to do so after an affordable conversion to XML.
Now i seek to build a set of queries to extract information from multiple converted eventlog files at once.
Are there people on this list who have experience or are open to building experience on this topic ?
Br,
Joris
Hey Christian Indeed, converted to xml first. BaseX absorbs the data well. The kind of queries I'll need to write may be quite complex. At least to me at this point, I noticed BaseX to have more under the hood i have yet to explore. Typically I expect to query for unique eventid to create a baseline and for one or more reoccurring field values which may span any or all eventid. Here i will need to do pattern matching for example for uuid, ip, fqdn etc. My preliminary super simple tests have shown this to be feasible. Based on these results and queries I'd the seek to export to json or other formats. Maybe also to send data to elasticsearch or neo4j. One other thing I'd like to explore is if I can query and/or index .evt and. evtx files directly by repurposing source code. Best Regards, Joris \-------- Oorspronkelijk bericht -------- Aan 12 apr. 2021 12:51, Christian Grün < christian.gruen@gmail.com> schreef:
Hi Joris,
Have you already exported the MS windows events to XML, and are you now trying to extract specific information from that files?
Best, Christian
On Wed, Apr 7, 2021 at 2:13 PM Joris Lambrecht <commandline@protonmail.com> wrote:
Dear,
For the longest time a good tool to datamine ms windows eventlogs escaped me.
BaseX appears to offer the toolkit which could permit to do so after an affordable conversion to XML.
Now i seek to build a set of queries to extract information from multiple converted eventlog files at once.
Are there people on this list who have experience or are open to building experience on this topic ?
Br,
Joris
Hi Joris,
The kind of queries I'll need to write may be quite complex. At least to me at this point, I noticed BaseX to have more under the hood i have yet to explore.
As XQuery has a much broader scope than SQL, you can write very sophisticated queries. In our documentation, you can find some simple use cases; see e.g. [1,2,3].
One other thing I'd like to explore is if I can query and/or index .evt and. evtx files directly by repurposing source code.
As far as I know, these formats are proprietary, but you could include external converters for that [4]. Such a converter could also be invoked from XQuery [5], and the result could be further processed. Hope this helps, Christian [1] https://docs.basex.org/wiki/XQuery_3.0 [2] https://docs.basex.org/wiki/XQuery_3.1 [3] https://docs.basex.org/wiki/XQuery [4] https://www.vanimpe.eu/2015/07/16/use-evtxparser-convert-windows-event-log-f... [5] https://docs.basex.org/wiki/Process_Module
Typically I expect to query for unique eventid to create a baseline and for one or more reoccurring field values which may span any or all eventid. Here i will need to do pattern matching for example for uuid, ip, fqdn etc.
My preliminary super simple tests have shown this to be feasible.
Based on these results and queries I'd the seek to export to json or other formats. Maybe also to send data to elasticsearch or neo4j.
One other thing I'd like to explore is if I can query and/or index .evt and. evtx files directly by repurposing source code.
Best Regards,
Joris
-------- Oorspronkelijk bericht -------- Aan 12 apr. 2021 12:51, Christian Grün < christian.gruen@gmail.com> schreef:
Hi Joris,
Have you already exported the MS windows events to XML, and are you now trying to extract specific information from that files?
Best, Christian
On Wed, Apr 7, 2021 at 2:13 PM Joris Lambrecht <commandline@protonmail.com> wrote:
Dear,
For the longest time a good tool to datamine ms windows eventlogs escaped me.
BaseX appears to offer the toolkit which could permit to do so after an affordable conversion to XML.
Now i seek to build a set of queries to extract information from multiple converted eventlog files at once.
Are there people on this list who have experience or are open to building experience on this topic ?
Br,
Joris
participants (3)
-
Christian Grün -
commandline-be -
Joris Lambrecht