January 21, 2008
12:20
by: Dan

Debugging provider crashes on SFCB with gdb

I’ve never had to debug a multi-process program with GDB before.  Mostly because I do a lot of my debugging with printf() and elbow-grease.  However, a CIM provider is in an environment where a crash happen within the broker, which means you can’t (easily) add print statements to the appropriate place.  If you pass something broken to a CBFoo() call, the CIMOM will crash a ways down the line.

With Pegsus, you can turn off provider processes, which forces thread-only mode.  This means that the whole CIMOM is vulnerable to a bad provider, but it also means that running the cimserver in gdb is easy and straightforward.  Doing the same in SFCB, however, is not as simple.  Your bad provider ends up in a child of the main process, which crashes without GDB’s full attention, which isn’t very helpful.

While helping to debug a particularly nasty crash with an Indication provider, I learned a (relatively) easy way to catch the crash with GDB.  First, I started SFCB and subscribed to the indication.  This forked off a process to host the provider.  While in the simple case, the PID of the new process will be the highest-numbered sfcbd process, you can also find out which one it is with something like this:

% PROV=libVirtComputerSystemIndication
% for i in $(ps ax | grep sfcbd | awk ‘{print $1}’Smilie: ;); do
> grep -q $PROV /proc/$i/maps && echo $i;
> done

That checks all of the currently running sfcbd processes to see which has your provider loaded and prints the PID.  Next, I attached to the process with gdb and allowed it to continue:

% S gdb /usr/local/sbin/sfcbd
GNU gdb 6.6
Copyright (C) 2006 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type “show copying” to see the conditions.
There is absolutely no warranty for GDB.  Type “show warranty” for details.
This GDB was configured as “x86_64-suse-linux”…
Using host libthread_db library “/lib64/libthread_db.so.1”.
(gdb) attach 5832
<snip>
(gdb) c

Next, I triggered the crash and GDB caught it, allowing me to get a stack trace and examime the situation a little.  After that, figuring out the issue was easy.

Posted in Codemonkeying Tagged , ,

January 11, 2008
17:11
by: Dan

S/PDIF and Dolby Digital out from MythTV

One of the nice things about getting the upstairs movie room finally bootstrapped is the return to having a real Dolby Digital receiver and more than just stereo sound.  I’ve always been a big fan of loud bass-booming movies with surround sound, but we decided to put the receiver in a closet until the upstairs was ready, instead of sqeezing it into the living room.

The last time I was really using a receiver, I didn’t have an HDTV, and thus didn’t have an HD capture card for Myth.  This meant that analog stereo out to the receiver was about as good as anything.  At the time, I didn’t have a Myth machine fast enough to reliably do DVDs, so digital audio was pointless.

Yesterday I set out to get S/PDIF output working from the Myth machine to the receiver.  My Myth machines all netboot a read-only copy of FC6 from a server on the network, so upgrading too much software and certainly bumping to a newer version of Fedora is a bit of an ordeal; I was hoping that FC6 would have everything I needed.

I started reading about digital sound with myth, and decided that most of the configuration should be done for me already.  I downed the box, and put in a spare SoundBlaster Live! card that I had lying around.  The card has a “digital output” jack on the back, which I connected to the Coaxial digital input of the receiver with a mono headphone-to-RCA adapter.

If I used something like mplayer to just play audio files (or a DVD) directly, I could see digital audio going to the receiver, but only stereo, and not Dolby Digital.  I think this is to be expected, as I wasn’t telling mplayer to pass the AC3 stream directly to the card.  Thus, alsa, or mplayer, or something, was dumping the regular audio out over the digital connection.  A good start, but not quite enough.

So, after reading a howto, I tried the following command:

mplayer -ac hwac3 -ao alsa:spdif dvd://

From this, I would get nothing.  Not even regular 2-channel audio.  Not even a blip on the receiver that something had happened on the digital audio line.  So I played around with things, including the mixer, and lots of mplayer settings.  I couldn’t figure out why the raw dump of the AC3 stream wasn’t working.  In my frustration, I tried turning back on the integrated sound, since ALSA claimed that it had a subdevice for IEC958 (S/PDIF) audio.  However, pointing mplayer at that resulted in an error, instead of just the appearance of playback without any actual sound (which is what the SoundBlaster was doing).

I put the SoundBlaster back in and fiddled some more.  I could see that the subdevice listing showed the analog surround output jacks, but no IEC958 like the integrated audio had:

[root@video2 ~]# aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Live [Dell Sound Blaster Live!], device 0: emu10k1x [EMU10K1X Front]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Live [Dell Sound Blaster Live!], device 1: emu10k1x [EMU10K1X Rear]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 0: Live [Dell Sound Blaster Live!], device 2: emu10k1x [EMU10K1X Center/LFE]
Subdevices: 1/1
Subdevice #0: subdevice #0

I figured that perhaps one of these subdevices was actually responsible for the digital audio, but perhaps ALSA wasn’t detecting it.  So, I started pointing mplayer directly at the subdevices hoping to get a result:

mplayer -ac hwac3 -ao alsa:device=hw=0.2 dvd://

Finally, I saw the blue “Multichannel Decoding” light illuminate on the receiver and the Dolby Digital logo appear!  The receiver reported 6.1 channel decoding and everything.  I was curious if I had just picked the right subdevice right off, so I tried hw=0.1 and hw=0.0, and both worked just the same.  Confused about this, I tried without a subdevice using just hw=0 and that also worked.  I then thought that perhaps it was the ALSA aliases that were misdirecting the audio, but device=default worked as well.  So, it appears that only the spdif alias was wrong, and that unless you tell mplayer otherwise, it tries that alias.

So, I set off to get Myth setup to pass the AC3 from HD programs through to the card in a similar fashion.  Analog recordings already worked by just sending the stereo audio out the digital output.  To configure the passthrough, I went into Setup->Setup->General, and on the third page, I enabled AC3 and DTS to SPDIF passthrough.  I also left the passthrough device as “Default”, even though the SPDIF device was an option.

After these changes, I can watch HD programs and DVDs through Myth with full 6.1 Dolby Digital or DTS surround sound!

Posted in Linux Tagged ,

January 9, 2008
17:13
by: Dan

libvirt-cim in Fedora

Today the first build of libvirt-cim hit the Fedora development tree.  This means that by just running “yum install libvirt-cim“, you can have a working CIM provider for Xen on your Fedora 8 machine (assuming you have the development repository enabled).  After installing the provider, you can do something like the following to check that it’s working:

[root@f8test ~]# wbemcli ein http://root:password@localhost/root/virt:CIM_System
localhost:5988/root/virt:Xen_HostSystem.CreationClassName=”Xen_HostSystem”,Name=”f8test.danplanet.com”
localhost:5988/root/virt:Xen_ComputerSystem.CreationClassName=”Xen_ComputerSystem”,Name=”Domain-0″

You’ll get a Xen_ComputerSystem object for each of the domains you have on the system, and a Xen_HostSystem object for the host itself.

The build contains a snapshot of the development tree from a few days ago, and looks very close to what version 0.1 will be.  There is a pretty substantial amount of support for Xen paravirt guests, although there are still a few rough spots.  After we snap version 0.1, we will focus on improving support for Xen fullvirt and KVM guests, which should be very similar.  Right now, you can do things like enumerate KVM guests and devices, but you can’t do much more than that.

If you try it out (successfully or not), be sure to drop by the libvirt-cim mailing list and let us know!

Posted in Codemonkeying, Linux Tagged , ,

January 7, 2008
17:18
by: Dan

AGP + PCIe = Bad news

While the title seems gloomy, this post is actually very good news for my TV addiction.

So, in an attempt to redistribute video capabilities in the house to accommodate the new TV, I decided to reprovision my second-best MythTV frontend box and use a MediaMVP (with mvpmc of course) in it’s place for the time being.  This will only be a temporary solution, but since we only have a standard-def TV in the bedroom, it seemed like the best move.

So, I took the box upstairs, VGA cable in hand, excited about finally having a VGA port that can take 1080p input.  I plug in the box and boot it up, and it detects the correct resolution automatically (thanks nVidia!) and presents me with a MythTV menu.  Before I go on, I better give a little background on my current HD Myth setup.

Before Sunday morning, the only HD set in the house was my 40″ JVC LCD.  Connected to it, I have a 2.8GHz Pentium4 with a 256MB nVidia 6200 AGP card in it.  I use a DVI-HDMI converter to drive the TV.  Until today, this box has been just a little too slow to play HD content without the help of XvMC (or so I thought).  XvMC is nice if your system is too slow, because the GPU can do some of the MPEG decoding for you.  However, it brings some other issues, like monochrome On-Screen Display and really choppy playback while the OSD is on the screen.  The result was that I had an ugly black-and-white OSD while watching HD, and I always had to hit ‘exit’ immediately after invoking any function that brought up the OSD.  However, it worked and I just planned to upgrade the box later.

So, back to the “second-best” box upstairs.  I plug it in, it drives the new TV to 1080p, and I go to watch an HD program just to see how underpowered it is.  What, you ask, happened?  It played the HD content perfectly, with no XvMC, and with a full-color OSD.  Huh?  What is going on here?  This box is clearly slower than the other in every respect.

So, I started thinking about what is faster about this box than the other.  Both had Intel processors, both were Intel chipsets, both were using similar AGP cards, with the same driver version.  Then something I read a few weeks ago popped into my head.  I read somewhere that motherboards were not supposed to support AGP and PCIe slots because of some quirk about how they would be presented to the system.  I remembered that the (ultra-cheesy) motherboard that I had in the “fast” box actually had both types of slots, since it was purchased just as PCIe was gaining traction.  I figured that maybe the AGP slot was crippled (I think the article mentioned that the AGP slot was hung off of a slower-than-AGP PCIe drop) and that was causing me not to be able to push enough data to the card to support HD content.

So, I ripped a PCIe 256MB nVidia 6800 from another box and swapped the AGP card out.  I booted up, turned off XvMC and pulled up an HD program.  It played perfectly!  I’m now kicking myself for not thinking to try this earlier, instead of dealing with a monochrome, jumpy OSD for a year.  The moral of the story is: Don’t buy a combo AGP+PCIe motherboard, and if you do, don’t expect AGP to work well.

Posted in Linux Tagged

January 6, 2008
17:52
by: Dan

I may have ended the Hi-Def disc format war

I know, it sounds silly, but let me lay it out:

Friday, I go into the local VideoOnly store and purchase a big-screen TV.  This is the first step in furnishing the last room in our house, which we want to be our primary movie-watching space.  I went in hunting for a Sony KDS-60A3000.  They didn’t have any in stock (which didn’t really matter to me, as I have no couch to set in front of anyway), but the salesperson convinced me that for the room size, something a little smaller would be more appropriate.  I struggled with the “get a smaller TV” concept, but I had read previously that a 60″ would be about the largest I should get.  So, we ended up with a Samsung HL-5676S, a 56″ model.  We couldn’t get it into the car (a good sign that I hadn’t settled for too small of a screen), so delivery was set for Sunday.

Okay, back to me catalyzing the end of the format war.  Early Saturday morning, someone very high up in Warner Bros. obviously noticed my purchase.  Knowing that in about 24 hours, I would plug in a paltry standard-def DVD player into my large screen and be utterly horrified, they decided that they better end the war before I had to bring myself to a format decision.  Thanks Warner!

Come Sunday, the TV was delivered.  The size seemed fine, and I had gotten over this “smaller TV” stuff.  We plugged it in and brought something up before the delivery people left to make sure it worked.  Very shortly after that, I noticed that something wasn’t right.  If I bring up the TV’s menu (which is a horizontal bar across the bottom of the screen), I see that the entire picture is very clearly rotated counter-clockwise about 2 degrees.  We measured the distance from the bottom of the bar to the top of the bezel, and it was at least a half-inch greater on the right side!  This was also noticeable on letterbox DVDs and anything that had large horizontal or vertical lines in the picture.  What the heck?

I figured that, being a projection screen and all, that there must be an adjustment of some kind for this.  I googled for “samsung dlp rotate” and, to my dismay, found only reports of similar issues.  Is this some systemic issue with the new slim line of Samsung DLP TVs?  With all the other reports, I wasn’t interested in playing swap with the store until I got one that was straight and true.  I called the guy at VideoOnly and said I wanted to swap for the Sony.  Luckily, VideoOnly is awesome and they took care of it and said I could keep the Samsung until the Sony arrived.

Okay, so my little purchase probably didn’t end the format war, but after looking at the utterly poor image from my standard-def player on the big screen, I’m glad that things may be starting to move on that front.  I think I wanted HD-DVD to win, because I don’t like Sony’s obsession with proprietary formats (across all technologies).  However, the good news is that either Sony or Microsoft gets screwed either way, so an end in either form seems like a good thing.

Posted in Hardware

December 12, 2007
17:58
by: Dan

Testing with pywbem

Recently a colleague pointed out that our VirtualSystemManagementService’s RemoveResourceSettings method (in libvirt-cim) was marked with the wrong type.  This was done early on, and probably was just a mistake, given that AddResourceSettings and ModifyResourceSettings both took the same type: an EmbeddedObject.  RemoveResourceSettings takes a reference, since anything you could possibly delete would be representable as a reference.  It’s the right way, but annoying, given that I had already built a common resource management infrastructure around the idea of having a list of resource objects to (Add,Modify,Remove).

Anyway, I coded up the change to the provider itself and set out to test it.  I quickly realized that I didn’t know what the CIMXML for a method call with an array of references looked like.  Up to this point, I have been unit testing the method providers by just hacking up a CIMXML template and stuffing it into wbemexec or wbemcat.  It sucks, but it works.

However, this time I was stuck.  I didn’t have an example, and some guessing didn’t work in a short amount of time.  I decided to try to work up something in pywbem to see if I could get not only the method call made for testing, but also a glimpse at what the CIMXML for it looked like.  I ended up with this very small bit of python code that was extremely helpful:

from pywbem import WBEMConnection, CIMInstanceName

c = WBEMConnection(“http://localhost”,
(“root”, “password”Smilie: ;),
“root/virt”Smilie: ;)
c.debug = True

r = CIMInstanceName(“Xen_DiskResourceAllocationSettingData”,
keybindings={“InstanceID”:”pv0/xvda”})

res = c.InvokeMethod(“RemoveResourceSettings”,
“Xen_VirtualSystemManagementService”,
ResourceSettings=[r])

print c.last_request

This connects to localhost, creates a reference, and makes the method call with that reference as a single-element array.  After the operation, the “last_request” field contains the CIMXML it sent to the server, but only if the “debug” property is set to True (something the documentation conveniently leaves out).  The result (with the boiler-plate cut out) looks like this:

<PARAMVALUE NAME=”ResourceSettings” PARAMTYPE=”reference”>
<VALUE.REFARRAY>
<VALUE.REFERENCE>
<INSTANCENAME CLASSNAME=”Xen_DiskResourceAllocationSettingData”>
<KEYBINDING NAME=”InstanceID”>
<KEYVALUE VALUETYPE=”string”>
pv0/xvda
</KEYVALUE>
</KEYBINDING>
</INSTANCENAME>
</VALUE.REFERENCE>
</VALUE.REFARRAY>
</PARAMVALUE>

Not that I really need it at this point, given that the pywbem case was so easy to write.  However, it provides a good example of how to not only use pywbem to figure out the proper CIMXML for an operation, but also debug it if it’s wrong.

Posted in Codemonkeying Tagged ,

December 3, 2007
16:48
by: Dan

J-Pole

A while back, I built a J-Pole (aka “Copper Cactus”Smilie: ;) antenna for the 2-Meter band.  The intent was to put it up in the peak part of the attic, which would be the highest spot possible without getting it above the roof line.  Not thinking things through as usual, it ended up being too tall to fit there. I ended up just putting it down at the second-story floor level, in one of the unfinished storage areas.

I ordered a bunch of LMR-400 (aka “the good stuff”Smilie: ;) line to feed it from the office downstairs (directly below said storage area).  This weekend I got around to running the line and mounting the antenna in somewhat of a formal fashion.  Until now, I had it clamped in place, fed with a 50′ length of pre-made RG-58U (aka “the bad stuff”Smilie: ;).  Turns out, the line loss was so bad (about 2.8 db one-way I think) that the SWR always showed about 1:1 and there was no tuning to be had, presumably because the line was just losing it all by the time it got back to the radio.

Switching to the LMR-400 was a huge difference.  The loss with it should be about 0.8 db.  Immediately upon testing at 146.0 MHz, the SWR was about 2.3:1 with 25 watts.  After about 30 trips upstairs and back, I got the SWR down to 1.2:1 at 148 MHz and about 1.3:1 at 144 MHz.  Not too bad for my first J-Pole!

After I get the radio put back in Taylor’s car, I hope to be able to really test this on 2-Meter simplex.  Previously, I was able to get a good signal out to her with 2.5 or 5 watts as soon as I could hear her (when she was using 50 watts and a small antenna).  Hopefully, this will extend my receive range, making her 50 watts a little more effective.

Posted in Radio Tagged ,

December 1, 2007
08:48
by: Dan

Now available in rawhide: libcmpiutil

Yesterday we snapped the 0.1 release of libcmpiutil and I completed the Fedora packaging process.  This involved getting a CVS module created, importing the package, and requesting a build in Koji.  All in all, I found the procedure to be pretty impressively automated.  The result is that this morning, libcmpiutil-0.1 is filtering out to fedora development mirrors!

Next stop: libvirt-cim!

Posted in Codemonkeying

November 28, 2007
15:19
by: Dan

Easier association providers with libcmpiutil

In CIM, associations are the powerful links between instances that model relationships.  A particular association may relate any number of objects to any number of other objects.  This can make implementation of the association confusing, difficult, and error-prone.

Take, for example, the CIM_ElementCapabilities association.  For an SVPC (Virtualization) provider, you could resolve the association against either half of any of the following pairs of related classes:

  • CIM_ComputerSystem, CIM_EnabledLogicalElementCapabilities
  • CIM_System, CIM_VirtualSystemManagementCapabilities
  • CIM_ResourcePool, CIM_AllocationCapabilities

That can become a lot of work because the provider only gets the association class name (CIM_ElementCapabilities) and the reference passed from the client to do its work.  It seems to be standard practice to solve this problem by subclassing the association superclass for each possible combination of related classes which helps to split up the implementation.  However, this (in my limited experience) tends to result in a lot of duplicated effort, more actual provider modules, etc.  Further, an association provider must implement all four intrinsic methods: Associators(), AssociatorNames(), References(), and ReferenceNames().  Finally, the provider must check the Role, ResultRole, ResultClass, and AssociationClass parameters specified by the client and filter the results of the association appropriately for each of these operations.  If you subclass the association for each pair, even reusing as much code as possible in each provider, you still end up with what I would call a mess.

While developing libvirt-cim, I decided to try to come up with an abstract representation of a single association that relates two specific types.  What I ended up with was the std_association module in libcmpiutil.  This allows you to specify the characteristic details of a particular association (the two class types being associated) and a handler function to do the matching.  The infrastructure does the argument checking, handler dispatch, and result filtering automatically.

Continue reading →

Posted in Codemonkeying Tagged

November 20, 2007
15:50
by: Dan

Mock Rocks!

I’ve been working through the process of getting a libcmpiutil package built for Fedora 9.  I’ve never packaged something for a distro before, so some of the things are a bit new to me.  Luckily, Daniel Veillard did a good chunk of the up-front work of writing the spec files; only minor tweakage was needed.

I’ve been using Fedora 8 for a development machine, because the Pegasus package is fine for what we need (aside from the schema changes).  However, during the review process, an issue came up which was no doubt related to a newer compiler version in the -development head of Fedora (what will become Fedora 9).  I, of course, did not see these issues on my Fedora 8 machine, and needed a way to test my fixes before posting for review again.

I considered installing a Fedora 8 VM, enabling the fedora-development repository and upgrading everything.  However, our lab was being really slow today, and accessing my KVM-capable machines was just not going to happen.  Knowing that Mock is supposed to help this sort of thing, I decided to give it a shot.

After setting up a build user (per the MockTricks page), I can do this:

% sudo su – build
% mock -r fedora-devel-x86_64 rebuild libcmpiutil-0.1-4.src.rpm

Not only do I get a Fedora 9 build environment, but I get my build dependencies checked and a .fc9 SRPM out of the whole deal!  It’s really cool to be able to get an automated build environment without needing to hose a machine with a test installation.  You can even test 32-bit builds on a 64-bit machine with minimal effort.  Finally, if you need to do something other than build an SRPM in the sandbox, you can get a chroot to any of the environments you need like this:

[build@guaranine ~]$ mock -r fedora-devel-x86_64 shell
INFO: mock.py version 0.8.7 starting…
State Changed: init plugins
State Changed: start
State Changed: lock buildroot
State Changed: clean
State Changed: init
State Changed: lock buildroot
State Changed: enabled yum cache, cleaning yum metadata
State Changed: enabling root cache
State Changed: unpacking cache
mock-chroot> cat /etc/fedora-release
Fedora release 8.90 (Rawhide)
mock-chroot>

The one thing I haven’t figured out how to do yet is test build a dependent set of RPMs (such as libcmpiutil and libvirt-cim), but I feel there must be a way.

Posted in Codemonkeying, Linux Tagged ,