XML not fully escaped
jantman opened this issue · comments
Hello, I'm seeing some issues with broken XML due to ASCII control characters in my test output. This sounded a bit like #5 to me, but that says that such issues should be fixed with Builder 3.1.1, and my bundle is using builder 3.2.2.
The specific issue I'm having is with acceptance tests that shell out to execute commands, which include a progress bar. The results.xml that I get includes something like:
<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="rspec" tests="5" failures="3" errors="0" time="65.846183" timestamp="2015-05-09T09:54:30-04:00">
<properties/>
<testcase classname="spec.acceptance.vagrant-r10k.vagrant-r10k_spec" name="provider/virtualbox/vagrant-r10k it should behave like provider/vagrant-r10k module path different from Puppet provisioner skips r10k deploy" time="12.292970">
<failure message="some properly-escpaed text here">
<
Hi there! Raw bytes outside the closing triad (]]>) are valid and legal inside CDATA sections of XML. What is having trouble reading your file?
Did you manually truncate your example? It seems to be missing the closing triad.
Also, I copy/pasted your example into a file, added the closing triad, and it parses successfully with Nokogiri:
irb(main):003:0> Nokogiri::XML.parse(File.read("tmp/test.xml"))
=> #<Nokogiri::XML::Document:0x3fc4d5f795fc name="document" children=[#<Nokogiri::XML::Element:0x3fc4d5f792dc name="testsuite" attributes=[#<Nokogiri::XML::Attr:0x3fc4d5f79278 name="name" value="rspec">, #<Nokogiri::XML::Attr:0x3fc4d5f79264 name="tests" value="5">, #<Nokogiri::XML::Attr:0x3fc4d5f79250 name="failures" value="3">, #<Nokogiri::XML::Attr:0x3fc4d5f7923c name="errors" value="0">, #<Nokogiri::XML::Attr:0x3fc4d5f79228 name="time" value="65.846183">, #<Nokogiri::XML::Attr:0x3fc4d5f79214 name="timestamp" value="2015-05-09T09:54:30-04:00">] children=[#<Nokogiri::XML::Text:0x3fc4d5f783b4 "\n ">, #<Nokogiri::XML::Element:0x3fc4d5f78300 name="properties">, #<Nokogiri::XML::Text:0x3fc4d5f78148 "\n ">, #<Nokogiri::XML::Element:0x3fc4d5f78094 name="testcase" attributes=[#<Nokogiri::XML::Attr:0x3fc4d5f78030 name="classname" value="spec.acceptance.vagrant-r10k.vagrant-r10k_spec">, #<Nokogiri::XML::Attr:0x3fc4d5f7801c name="name" value="provider/virtualbox/vagrant-r10k it should behave like provider/vagrant-r10k module path different from Puppet provisioner skips r10k deploy">, #<Nokogiri::XML::Attr:0x3fc4d5f78008 name="time" value="12.292970">] children=[#<Nokogiri::XML::Text:0x3fc4d5f657a0 "\n ">, #<Nokogiri::XML::Element:0x3fc4d5f656ec name="failure" attributes=[#<Nokogiri::XML::Attr:0x3fc4d5f65688 name="message" value="some properly-escpaed text here">] children=[#<Nokogiri::XML::Text:0x3fc4d5f65250 "\n ">, #<Nokogiri::XML::CDATA:0x3fc4d5f6519c "expected command to exit with 0 but got exit code: 1\n\nstdout: You appear to be running Vagrant outside of the official installers.\nNote that the installers are what ensure that Vagrant has all required\ndependencies, and Vagrant assumes that these dependencies exist. By\nrunning outside of the installer environment, Vagrant may not function\nproperly. To remove this warning, install Vagrant using one of the\nofficial packages from vagrantup.com.\nBringing machine 'default' up with 'virtualbox' provider...\n default: vagrant-r10k: module_path \"puppet/NOTmodules\" is not the same as in puppet provisioner; not running\n==> default: Importing base box 'vagrantr10kspec'...\n\n�[KProgress: 10%\n�[KProgress: 20%\n�[KProgress: 30%\n�[KProgress: 50%\n�[KProgress: 70%\n�[KProgress: 90%\n\nstderr: There was an error while executing `VBoxManage`, a CLI used by Vagrant\nfor controlling VirtualBox. The command and stderr is shown below.\n">, #<Nokogiri::XML::Text:0x3fc4d5f650e8 "\n ">]>, #<Nokogiri::XML::Text:0x3fc4d5f64f44 "\n ">]>, #<Nokogiri::XML::Text:0x3fc4d5f64da0 "\n">]>]>
Perhaps copy/paste isn't preserving the exact encoding. If you could upload your xml file somewhere that would be helpful.
Thanks so much for responding quickly, and sorry for any confusion. Yes, what I pasted was a snippet from the actual file; the trouble reading it was with Firefox and Chrome (I was trying to use Ant's XSL to view the JUnit locally), and also with xmllint reporting all sorts of errors, specifically including PCDATA invalid Char value 27.
I've uploaded an actual output file to https://gist.github.com/jantman/7b29c8aeff31cdb1799a ; I downloaded the Raw version of that Gist and was able to reproduce the errors, so it seems to be preserving the encoding correctly.
As far as I can tell, the only issue are the ASCII ESC's (\033). The following code renders the file completely readable, and it now passes xmllint with no warnings:
content = File.read('results.xml')
content.gsub!(/\033/, "\uFFFD")
File.open('results.xml', 'w') { |file| file.write(content) }
I have faced this issue too. Any plans on fixing this? Can I do a PR escaping it here instead of builder gem?
This was planned to be fixed in builder gem with this a long time time ago, but the issue was closed for no reason - jimweirich/builder#3.
@faucct Jim sadly passed away a little while ago. It's possible nobody has taken over the gem and they simply closed everything.
If you'd like to replace invalid characters in here instead that'd be fine, too.
But yes, there are some characters which xml is incapable of representing, including control codes. They will just never work.
Hi there! 👋 I was totally wrong and will eat my own words. I've updated master to do simpler gem-less XML output which is a little more proactive about escaping and sanitizing content. Please try the latest pre-release (gem install --pre rspec_junit_formatter or in your Gemfile gem "rspec_junit_formatter", ">= 0.3.0.pre") and let me know if you're still having trouble. Thanks!