Rebuilding Yum Database

When starting Hadoop components on the Linux virtual machines on my laptop, I periodically get failures at startup. The culprit? A corrupt yum database. I’ve been running Hortonworks 2.0 recently and have used the Ambari GUI to start services. The failure I encounter looks like this:

notice: Finished catalog run in 0.65 seconds
err: /Stage[1]/Hdp/Hdp::Lzo::Package[64]/Hdp::Package[lzo 64]/Hdp::Package::Process_pkg[lzo 64]/Package[lzo]/ensure: change from absent to present failed: Execution of '/usr/bin/yum -d 0 -e 0 -y install lzo' returned 1: rpmdb: Thread/process 3570/140380093466368 failed: Thread died in Berkeley DB library
error: db3 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db3 -  (-30974)
error: cannot open Packages database in /var/lib/rpm
CRITICAL:yum.main:

Error: rpmdb open failed

The following series of commands straighten out the situation I typically encounter.

# rm -f /var/lib/rpm/__db*
# db_verify /var/lib/rpm/Packages
# rpm --rebuilddb
# yum clean all

The command yum clean all will not work until the first 3 commands are run, should the case be the DB_RUNRECOVERY error show above.

vmware fusion fixed ip stuff

As a presales engineer at a small big data company with our own database that runs on many platforms, I now use VMWare Fusion all the time.

Here’s some of the grunt work I do when creating new VMs for which I want a fixed IP address and a changed hostname.

If I make the new VM and it’s linux, I may use the VMWare rapid installation feature. Then I want to change the root password. That just requires a restart. During restart, hit any key on the keyboard to bring up the Linux GRUB screen. Then hit “e” on the keyboard to edit the start parameters. Use the down arrow to get to the line starting with “kernel”. Then hit e again to edit that line. Delete the last two words, “rhgb quiet” and replace them with “1”. Then hit enter to preserver your change and then b to boot. You will start in single user mode as root. From there, you can use the command “passed” to provide a new password for user root.

Then to change the hostname, I use these three steps on Centos Linux, after becoming root of course.
1. Use the hostname command to change the host name.
[root@localhost ~]# hostname yournewhostname.local

2. Edit /etc/sysconfig/network to include the new hostname.
NETWORKING=yes
HOSTNAME=yournewhostname.local

3. Back on my Mac, I figure out what the static IP address will be. With VMWare Fusion 5 (and apparently 4 or higher) this involves editing the following file:
/Library/Preferences/VMware Fusion/vmnet8/dhcpd.conf

Here’s a great write up on what to do.

4. Edit /etc/hosts to include the new hostname. Also include the static ip you chose in step 3.

5. I’ve been using Centos for my VMs. I’ve been using the GUI “Network Connections” tool under the System – Preferences menu to create the static ip address. On the first screen, select your ethernet card. Mine shows up as eth0 on the Centos Linux VMs I’ve created. Then click the Edit button.

On the resulting screen, I click on the IPv4 tab.

Then I switch the method to Manual. I click Add and make the address my new static IP that I chose in step 3 above. Netmask is 255.255.255.0. The gateway is the IP address of your Mac. I also make the DNS server the IP address for my Mac.

Using my digital camera with ubuntu (linux)

Working with my digital camera took a little doing on Ubuntu.

Ubuntu mounts it automatically, apparently using a gphoto2 library. So doing things via the GUI is really quite easy. Plug and play!

But I love the command line to manipulate files. I couldn’t find where the mount point was. Not under /mnt. Not under /media…

I learned that the gvfs-mount command shows information about what ubuntu GUI mounts automatically. And if you install gphoto2, you can run gphoto2 –auto-detect, which will show you the name of your camera and the USB info. Apparently the gphoto2 utility has a variety of camera manipulation commands it can perform.

It seems ubuntu puts samba at work behind the scenes to mount digital cameras, most of which have windows based file systems. Samba mounts to ~/.gvfs. So “ls ~/.gvfs yielded:
gphoto2 mount on usb%3A001,002

From there, I could run

ls ~/".gvfs/gphoto2 mount on usb%3A001,002"

Then I could see the DCIM directory. Off and running with commands like cp and rm!

ubuntu on my laptop, next steps

A quick followup. Ubuntu recognized my wireless Epson workforce 610 all-in-one printer. Prints fine. I haven’t tried to make any of the other functions work, like scanning. I see on the internet that some folks have had trouble with that.

Software updates for ubuntu work well. Using the gui Ubuntu software center, I downloaded and installed libreoffice’s database product called Base without a hitch. Then I used the command line tool called apt to download tcl/tk, sqlite3 and a bunch of database drives. That all worked easily and quickly.

Multi-media stuff like sound and video? No issues so far. Even burned some cds/dvds with Bracero.

Next up: Do some re-jiggering to make the laptop house multiple distributions. I’m reviewing How to install and boot 145 operating systems in a PC to help me plan things out. I’m only going with 4 to 6 distros, so should work just fine.

ubuntu on my laptop, so far

I took the plunge and updated my laptop from Vista to Ubuntu 11.10. I couldn’t eliminate a virus from the Vista deployment easily, didn’t want to spend money on commercial anti-virus, and learning linux better will help me at work. EMC bases many of their products on Linux (Avamar, Data Domain in the backup space, for example). I’m reasonably fluent in Linux, having installed it many times, worked with Oracle on it, used other unix flavours for DBA work.

How’d it go?

I have a compaq presario v6000 that uses a Broadcom wireless card and a NVidia graphics card.

I tried out the live-cd first, which allows you to run Ubuntu on your existing machine without writing to the hard drive so you can try things out without commitment. That could not display graphics correctly when run as is. Google searching taught me a fix. Hold down the F5 key during boot up and use the Other Options menu choice to select the boot option nomodeset. With that, live-cd display graphics correctly.

While running on live-cd, my wired network connection worked fine. But I couldn’t download the broadcom driver.

I decided to install. Install was smooth. The installer detected the broadcom wireless card and configured the drivers.

But after I installed ubuntu to the hard drive, I was back to the graphics problem. The hard drive install does not have the same menu choice via F5. What to do? I returned to google, and I learned that if you hold down the Shift key during boot, you can edit the grub boot up sequence. Using that technique, I again added nomodeset. I also learned that to make this change permanent, one needs to permanently edit the grub.conf file, which the shift key trick does NOT do.

Last issue. I wanted to use the Nvidia driver. To do that, I clicked on the system settings and chose additional drivers. This dialog displayed the Broadcom card configured during installation time, plus several Nvidia driver choices. I choose the recommended one. After doing this, I needed to reboot. Here I needed to hold down shift to turn off nomodeset in order to let Nvidia take over. Finally, I had to return to grub editing to do a removal of nomodeset.

Thoughts thus far: Ubuntu looks and works great. However, if you have a laptop like mine that needs tweaking, you still gotta be a geek to run linux.

Next: I will load up utlities like tcl/tk and sqlite, and possibly postgres to see how database stuff starts working. Will also see what end-user friendly databases are available for end users. Ubuntu 11.10 uses something called LibreOffice. Perhaps there is a Microsoft Access type tool in that.