Frebrilator
One of my favorite features in IIS 7.X/8.X is Failed Request Event Buffering, or FREB for short. Unfortunately, taking advantage of this feature does require certain preparations:
- The 'Tracing' role service of the Web Server role needs to be installed.
- You must enable Failed Request Tracing at the Web Site level
- You must define appropriate tracing rules at the application level
Any of these will require the application to be restarted. While not usually a problem it can be... inconvenient at times.
Frebrilator is just some sample code I've been putting together to scratch an itch I've been having for a while: Capturing FREB-like traces without using the normal means in IIS to do so.
FREB and ETW
FREB is built on top of the ETW (Event Tracing for Windows) infrastructure, around the following providers:
- {3a2a4e84-4c21-4981-ae10-3fda0d9b0f83} - IIS: WWW Server
- {AFF081FE-0247-4275-9C4E-021F3DC1DA35} - ASP.NET
- {06b94d9a-b15e-456e-a4ef-37c984a2cb4b} - IIS: Active Server Pages (ASP)
- {a1c2040e-8840-4c31-ba11-9871031a19ea} - IIS: WWW Isapi Extension
Most of these are 'classic' ETW providers, so they can be tricky to use correctly.
Since this was all based on ETW, I figured there should be a way to both capture usable ETW traces that contain the same data as that captured by FREB without having to use the FREB feature directly. Likewise, a way to convert those ETW traces back into per-request reports similar to those generated by FREB should be within the realm of possiblities. This is the purpose of Frebrilator.
What works
The current code has the following features implemented:
- The Microsoft.Diagnostics.Tracing.TraceEvent nuget package is used to capture and process ETL traces.
- Working TraceEvent parsers for the IIS, ISAPI, and ASP providers are included (generating these took quite a bit of effort).
- The -c switch can be used to capture an ETW trace from the 4 providers supported by FREB into an ETL file.
- The -p switch can be used to process an ETL file and generate one XML file per request into FREB format in the specified output directory.
The code is good enough to generate FREB files in XML format that can be visualized in Internet Explorer using the freb.xsl file included with IIS 7.X/8.X.
What is missing
There is still a lot of work that needs to be done to make this entirely useful:
- ETW traces don't appear to contain the original Level of the event (info, error, warning, etc.). Because of this, certain features of the freb.xsl stylesheet don't work (such as clearly highlighting errors). Note: I've now implemented a work-around for this using a static map, but there are chances events are missing or with the wrong name.
- There is data that will always be missing from the generated traces. For example, there is no way to resolve the authentication tokenUserName property, because only IIS at runtime has that information and it is not part of the captured ETW events.
- Work needs to be done to map certain fields used in the traces properly so that they match exactly what is produced by IIS natively.
- Currently, all generated XML files will contain as the machine name that of the computer where the ETL file is processed, rather than where it was captured (the ETL file itself doesn't contain that info).
- Need to add a command line option to capture a trace and produce the FREB files in real-time, instead of having to use an intermediate
Final notes
This is all just sample code. There are lots of things untested and working not quite right. All has been put together from looking over the IIS configuration files and generated FREB traces. Also would like to thank Vance Morrison for answering some questions regarding the TraceEvent library and ETW providers.