- 38% Python
- 30% Ruby
- 15% Perl
- 7% bash
- 7% VBA
- 0% Groovy
- 0% csh
Monday, September 14, 2009
POLL: If you could only have one scripting language? (Results)
POLL: How was your Fedora 11 Upgrade? (Results)
- 52% I use Ubuntu
- 19% Hell on Earth
- 19% I did a clean install
- 9% Easy and smooth
- 0% A little rough
Sunday, July 12, 2009
Fedora 11 - Upgrade from Fedora 10... not good... not good at all
Overview
So there I was, sitting at my workstation, doing a bit of work on my Fedora 10 server. When, all of a sudden a docile little dialog appeared notifying me that Fedora 11 was available and I could run pre-upgrade before upgrading.
Well, being an Ubuntu user, I have done many simple and smooth network based upgrades. So I figured, why not. Lets give it a shot. All my critical data is backed up anyway.
Now, I know what you are thinking. It states on the fedora project website that they prefer a clean install vs an upgrade. However, that doesn't mean an upgrade isn't possible? Right. I mean if your system pops a dialog saying "Distribution upgrades available" you should feel comfortable doing it. If it is possible but not recommended, why make the option so obvious.
I have been using Redhat/Fedora for as long as I have been using linux. Normally, I have followed the advice of doing a clean install vs upgrade. This time, I did the upgrade via network and let just say, it was not a good experience. Very little worked as advertised and I was always this close to system corruption and rebuild.
Now, I am not saying that an upgrade path from Fedora 10 to 11 spells doom. I am just saying that it almost did for me.
Background
My system has been upgraded twice by DVD. From Fedora 7 to 9. Then from Fedora 9 to 10. This may have set the stage for my upgrade distress. In all cases, there were some issues and bugs that I had to correct. Before FC7, I only did clean installs.
What Happened
My issues started during the pre-upgrade process. The updates dialog pops letting me know that FC11 is available. When clicked, the update process looks at your system and stages what it needs to upgrade.
During this phase, I got a message saying that the install image would not fit in /boot that that I needed a wired network after reboot to get the image during the upgrade. I had a wired network so I let it go.
After reboot, being sure to hit enter when you see the boot countdown (not documented) to get the option for the upgrade boot kernel.
Once I got into the actual upgrade process, I started getting error messages saying that it could not find packages (RPMs). That I could reboot or retry. The dialog said that reboot would mean possible corrption of my system. Retry, would simply retry (which is stupid). There was not a third option like, specify new location, data source, or continue anyway.
What is worse, is that it appeared that the installer was expecting RPMs to be in a different directory then they were in the install image (!!!). How this is possible, I have no idea since I never specified either. The update handled the mirror selection and all the behind the scenes work.
Well, corrupting my server, requiring a rebuild was just not going to happen. The following are a few thing I did to get through the mess and to a final working system.
Problem - Cannot Find Package to Install
It is possible that Fedora expected their install to barf so they were kind enough to provide most of the tools needed to correct them.
gives you a prompt. Here you will save the day.
gives you the current console log. Here you will see what is missing.
Two main tools that saved the day
Next thing you will need is a good working mirror. You will need to visit the fedoraproject site for that to find one. Each system will be different so you wil need to look at the console log to see where it is expecting to find the RPMs. Once you can see that (i.e. file:///mnt/system/...) you can cd to that directory and fetch each missing file one by one as you see the errors.
This job actually sucks since a missing gnome-libs problably means that gnome-libs-devel will be needed later. To get around this, I worked in broad strokes.
cat packages_wget.txt |grep gnome-libs |sh
Now the file (or any file with gnome-libs in it) will get fetched. This does not mean that it will get automatically installed but will be available for the installer.
I did this for all missing files. Later during the upgrade phase, there were more errors, so I followed the same approach.
I now what your are thinking
Why didn't I stop the install and download a DVD? I thought about that, however, the error messages dialog warned about corruption to the system and it was not clear how many files were actually missing. I might have been on my "last one". Once I got the above approach working, it went pretty fast. It was just an unforgiveable waste of time.
Hurray, the system upgrade is done. Reboot and Run Updates...!!!..Grumble
Finally, the upgrade completed. I rebooted the system and prepared for doing a yum update. This should be cut and dry now.... right? Nope. Aparently, there are several pre-release packages that were included in something the system was upgrading. Others had errors.
dhcp and cyrus-imap had a refernce to an openssl library that had an incorrect symlink. Looking at the error and correcting the symlink did the trick.
In a nutshell, yum update tried to upgrade a packages that does not exist on any mirror or even on the fedora site. Unless you know the full dependancy tree within the 631 updates, it is hard to turn off the ones in questions. For this reason, I had to upgrade in small batches until I found the packages that had the dependancies to the other packages that didn't exist.
Final Thoughts
I don't think I can stomach another Fedora upgrade. Will this happen to you? I am not sure. There could have been many reasons for my hardship, however, I have been using Fedora/Redhat for at least 10 years so I now the ins and outs. Install is not a strong point. It is flexible and easy to manage once running however. It is my (current) choice for a server OS.
My thought to you is: if the installer tells you that you do not have room on your /boot partition, cancel and walk away. I have a feeling that a DVD install would have gone better. A clean install would be best.
I just wish they could make it work like Ubuntu. That distro is great for package management. They just tend to be slower on updates (i.e. Java).
I will try to add more details about specific commands and syntax. My main goal here was to put information out, in the event, that other of you have run into the same issues.
Update 2009-Sept
Since my painful upgrade, the updates have been flowing just fine. My system has been running pretty well.
So there I was, sitting at my workstation, doing a bit of work on my Fedora 10 server. When, all of a sudden a docile little dialog appeared notifying me that Fedora 11 was available and I could run pre-upgrade before upgrading.
Well, being an Ubuntu user, I have done many simple and smooth network based upgrades. So I figured, why not. Lets give it a shot. All my critical data is backed up anyway.
Now, I know what you are thinking. It states on the fedora project website that they prefer a clean install vs an upgrade. However, that doesn't mean an upgrade isn't possible? Right. I mean if your system pops a dialog saying "Distribution upgrades available" you should feel comfortable doing it. If it is possible but not recommended, why make the option so obvious.
I have been using Redhat/Fedora for as long as I have been using linux. Normally, I have followed the advice of doing a clean install vs upgrade. This time, I did the upgrade via network and let just say, it was not a good experience. Very little worked as advertised and I was always this close to system corruption and rebuild.
Now, I am not saying that an upgrade path from Fedora 10 to 11 spells doom. I am just saying that it almost did for me.
Background
My system has been upgraded twice by DVD. From Fedora 7 to 9. Then from Fedora 9 to 10. This may have set the stage for my upgrade distress. In all cases, there were some issues and bugs that I had to correct. Before FC7, I only did clean installs.
What Happened
My issues started during the pre-upgrade process. The updates dialog pops letting me know that FC11 is available. When clicked, the update process looks at your system and stages what it needs to upgrade.
During this phase, I got a message saying that the install image would not fit in /boot that that I needed a wired network after reboot to get the image during the upgrade. I had a wired network so I let it go.
After reboot, being sure to hit enter when you see the boot countdown (not documented) to get the option for the upgrade boot kernel.
Once I got into the actual upgrade process, I started getting error messages saying that it could not find packages (RPMs). That I could reboot or retry. The dialog said that reboot would mean possible corrption of my system. Retry, would simply retry (which is stupid). There was not a third option like, specify new location, data source, or continue anyway.
What is worse, is that it appeared that the installer was expecting RPMs to be in a different directory then they were in the install image (!!!). How this is possible, I have no idea since I never specified either. The update handled the mirror selection and all the behind the scenes work.
Well, corrupting my server, requiring a rebuild was just not going to happen. The following are a few thing I did to get through the mess and to a final working system.
Problem - Cannot Find Package to Install
It is possible that Fedora expected their install to barf so they were kind enough to provide most of the tools needed to correct them.
Two main tools that saved the day
- wget
- lynx
Next thing you will need is a good working mirror. You will need to visit the fedoraproject site for that to find one. Each system will be different so you wil need to look at the console log to see where it is expecting to find the RPMs. Once you can see that (i.e. file:///mnt/system/...) you can cd to that directory and fetch each missing file one by one as you see the errors.
This job actually sucks since a missing gnome-libs problably means that gnome-libs-devel will be needed later. To get around this, I worked in broad strokes.
- I fetch a list of all the files using wget from the mirror site.
- I washed the "packages.html" file through cut and awk to get a list of only the rpm files
- This list of files was then washed through awk a second time to pre-pend wget http://..../file.rpm.
- This was then redirected into a text file for later use
cat packages_wget.txt |grep gnome-libs |sh
Now the file (or any file with gnome-libs in it) will get fetched. This does not mean that it will get automatically installed but will be available for the installer.
I did this for all missing files. Later during the upgrade phase, there were more errors, so I followed the same approach.
I now what your are thinking
Why didn't I stop the install and download a DVD? I thought about that, however, the error messages dialog warned about corruption to the system and it was not clear how many files were actually missing. I might have been on my "last one". Once I got the above approach working, it went pretty fast. It was just an unforgiveable waste of time.
Hurray, the system upgrade is done. Reboot and Run Updates...!!!..Grumble
Finally, the upgrade completed. I rebooted the system and prepared for doing a yum update. This should be cut and dry now.... right? Nope. Aparently, there are several pre-release packages that were included in something the system was upgrading. Others had errors.
dhcp and cyrus-imap had a refernce to an openssl library that had an incorrect symlink. Looking at the error and correcting the symlink did the trick.
In a nutshell, yum update tried to upgrade a packages that does not exist on any mirror or even on the fedora site. Unless you know the full dependancy tree within the 631 updates, it is hard to turn off the ones in questions. For this reason, I had to upgrade in small batches until I found the packages that had the dependancies to the other packages that didn't exist.
Final Thoughts
I don't think I can stomach another Fedora upgrade. Will this happen to you? I am not sure. There could have been many reasons for my hardship, however, I have been using Fedora/Redhat for at least 10 years so I now the ins and outs. Install is not a strong point. It is flexible and easy to manage once running however. It is my (current) choice for a server OS.
My thought to you is: if the installer tells you that you do not have room on your /boot partition, cancel and walk away. I have a feeling that a DVD install would have gone better. A clean install would be best.
I just wish they could make it work like Ubuntu. That distro is great for package management. They just tend to be slower on updates (i.e. Java).
I will try to add more details about specific commands and syntax. My main goal here was to put information out, in the event, that other of you have run into the same issues.
Update 2009-Sept
Since my painful upgrade, the updates have been flowing just fine. My system has been running pretty well.
Monday, June 29, 2009
Netbeans 6.7 Is Here
Today I downloaded and installed Netbeans 6.7. My first impressions are very good. The performance seems better and I really like what was done for unit testing.
I have used it under Windows and Linux. Thanks Netbeans team!! Another great version delivered right on time.
I have used it under Windows and Linux. Thanks Netbeans team!! Another great version delivered right on time.
Tuesday, June 23, 2009
HOWTO: Install Python Soap Libraries ZSI + SOAPpy + PyXML Under (of all things) Windows XP
Overview
I recently needed do some simple scripting using Python under Windows. The main goal was to test some web service calls (via SOAP). Python lets you put something like this together pretty quickly. Including WSDL processing, business logic, and unit testing.
Adding a SOAP library (ZSI+SOAPpy in this case) under linux (ubuntu or fedora) is also simple since you just install a package from a repo. However, I found that setting is up under Windows to be more of a pain.
Problem
Get ZSI, SOAPpy, fpconst, and PyXML setup for use in SOAP programming under Python 2.6.
The main issue is that one of the dependancies, PyXML, does not have a package for versions of Python > 2.4. This means that it will need to be manually built. The typical setup.py usually does the trick but for PyXML a C module was also included as part of the distro. Compiling modules under Windows requires an open source compiler or Visual Studio 2008. For me, neither are an option. Man I wish I could just use linux :-)
Lets Begin
After you fetch the packages through the Python Website...
Install fpconst
This, I think, is needed by SOAPpy. Fetch fpconst-0.7.2.tar.gz
Install PyXML
PyXML is required by SOAPpy and possibly ZSI. The problem you will run into installing it is that there are a few Python extensions that need compiled and unless you have Visual Studio 2008, it will not compile under Python 2.6.
Fortunatally, I discovered that SOAPpy (or ZSI) don't use these extensions so as long as you don't need it yourself you can install it as follows.
Installing SOAPpy
With PyXML installed, the hard part is over.
Install ZSI
Now the dependencies are install, ZSI can be installed. This is what you will use for all your SOAP calls. It is pretty well documented and fairly easy to use. The approach is similar to how it is handled in Java or .NET. Install is simple
Does it work?
To test, you will first need to process your web service WSDL file into something Python can work with. The approach is similar to what is done when you add a web service reference in .NET or JAVA. This this case, you use a simple command line utility called wsdl2py. On my windows computer, it is found in C:\Python26\Scripts\ You should find it in a similar place. Assuming that you do not have this script in the path and have not been clever enough to make a batch file, you can run it as follows. CD to the directory where you want your PY files created by the utility to go.
python C:\Python26\Scripts\wsdl2py -bu http://moneycow.sandwich.lan:8888/moneyfinder.asmx?WSDL
There are more parameters, but this command will get you what you need. You may see a few deprecation warnings but they don't hurt anything. In the end, you will get two files.
Unknowns
I have not figured out how to get SSL to work with ZSI. If any of you have figured this out, please post me a message. I would love to hear about it. It is not an issue for me now since none of the services I need to consume are Internet exposed. If I get stuck and need to access one, I will just use Java or .NET ;-)
Conclusion
If you have to use Windows and need to do Web Services with Python 2.6, the above steps should get you setup and ready to role. Once you are setup and get your first basic program working, you will find that Python does a good job and the performance is just fine.
I recently needed do some simple scripting using Python under Windows. The main goal was to test some web service calls (via SOAP). Python lets you put something like this together pretty quickly. Including WSDL processing, business logic, and unit testing.
Adding a SOAP library (ZSI+SOAPpy in this case) under linux (ubuntu or fedora) is also simple since you just install a package from a repo. However, I found that setting is up under Windows to be more of a pain.
Problem
Get ZSI, SOAPpy, fpconst, and PyXML setup for use in SOAP programming under Python 2.6.
The main issue is that one of the dependancies, PyXML, does not have a package for versions of Python > 2.4. This means that it will need to be manually built. The typical setup.py usually does the trick but for PyXML a C module was also included as part of the distro. Compiling modules under Windows requires an open source compiler or Visual Studio 2008. For me, neither are an option. Man I wish I could just use linux :-)
Lets Begin
After you fetch the packages through the Python Website...
Install fpconst
This, I think, is needed by SOAPpy. Fetch fpconst-0.7.2.tar.gz
- Under Windows I use WinRAR to extract the setup/temp folder
- run: python setup.py build
- run: python setup.py install
Install PyXML
PyXML is required by SOAPpy and possibly ZSI. The problem you will run into installing it is that there are a few Python extensions that need compiled and unless you have Visual Studio 2008, it will not compile under Python 2.6.
Fortunatally, I discovered that SOAPpy (or ZSI) don't use these extensions so as long as you don't need it yourself you can install it as follows.
- Under Windows I use WinRAR to extract the setup/temp folder
- run: python setup.py install --force -skip-build
Installing SOAPpy
With PyXML installed, the hard part is over.
- Unzip SOAPpy-0.12.0.zip to a setup/temp folder
- run: python setup.py build
- run: python setup.py install
Install ZSI
Now the dependencies are install, ZSI can be installed. This is what you will use for all your SOAP calls. It is pretty well documented and fairly easy to use. The approach is similar to how it is handled in Java or .NET. Install is simple
- WinRAR ZSI-2.0.tar.gz to a setup/temp folder
- run: python setup.py build
- run: python setup.py install
Does it work?
To test, you will first need to process your web service WSDL file into something Python can work with. The approach is similar to what is done when you add a web service reference in .NET or JAVA. This this case, you use a simple command line utility called wsdl2py. On my windows computer, it is found in C:\Python26\Scripts\ You should find it in a similar place. Assuming that you do not have this script in the path and have not been clever enough to make a batch file, you can run it as follows. CD to the directory where you want your PY files created by the utility to go.
python C:\Python26\Scripts\wsdl2py -bu http://moneycow.sandwich.lan:8888/moneyfinder.asmx?WSDL
There are more parameters, but this command will get you what you need. You may see a few deprecation warnings but they don't hurt anything. In the end, you will get two files.
- *_services.py
- *_services_types.py
Unknowns
I have not figured out how to get SSL to work with ZSI. If any of you have figured this out, please post me a message. I would love to hear about it. It is not an issue for me now since none of the services I need to consume are Internet exposed. If I get stuck and need to access one, I will just use Java or .NET ;-)
Conclusion
If you have to use Windows and need to do Web Services with Python 2.6, the above steps should get you setup and ready to role. Once you are setup and get your first basic program working, you will find that Python does a good job and the performance is just fine.
Subscribe to:
Posts (Atom)