Alasdair on Everything   

Solaris, Ruby and iconv.

January 30th, 2009

Solaris 10 has a native C library version of iconv. Unfortunately it lacks various features people might want to use, such as transliteration. You can see an example of this here:

# ruby -e "require 'iconv'; p Iconv.iconv('ascii//ignore//translit', 'utf-8', 'Tést')"
-e:1:in `iconv': invalid encoding ("ascii//ignore//translit", "utf-8") (Iconv::InvalidEncoding)
        from -e:1

Other people have commented on the problem, for example on this blog here. One suggestion is to install SUNWgnu-libiconv and utilise LD_PRELOAD. The SUNWgnu-libiconv package isn’t around on Solaris 10 (that I know of), so we need to compile up our own libiconv. And LD_PRELOAD is an inelegant, somewhat difficult to implement and potentially dangerous environment variable.

This sadly means we need to recompile things from source. But hey, it’s Solaris! I’m sure you’re used to this by now. So it’s time to get your recompiling hat on and build libiconv and Ruby. A fairly generic build would go something along these lines (feel free to alter all the “/opt/local” paths to suit your own requirements:

# mkdir -p /opt/src
# cd /opt/src
# wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.12.tar.gz
# gtar -zxf libiconv-1.12.tar.gz
# cd libiconv-1.12
# ./configure --prefix=/opt/local
# gmake
# gmake install

# cd /opt/src
# cat <<EOF > /tmp/ruby.patch
--- ruby-1.8.6-p287.good/ext/iconv/extconf.rb   Mon Feb 12 23:01:19 2007
+++ ruby-1.8.6-p287/ext/iconv/extconf.rb        Thu Jan 29 19:07:45 2009
@@ -22,6 +22,7 @@
       end
     end
     $defs.push('-DICONV_INPTR_CONST')
+    $DLDFLAGS += ' -L/opt/local/lib -R/opt/local/lib -liconv'
   end
   if conf
     prefix = '$(srcdir)'
--- ruby-1.8.6-p287.good/ext/iconv/iconv.c      Thu Jan 29 19:10:58 2009
+++ ruby-1.8.6-p287/ext/iconv/iconv.c   Thu Jan 29 18:20:56 2009
@@ -16,7 +16,7 @@

 #include "ruby.h"
 #include <errno.h>
-#include <iconv.h>
+#include </opt/local/include/iconv.h>
 #include <assert.h>
 #include "st.h"
 #include "intern.h"
EOF
# wget ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.7-p72.tar.gz
# gtar -zxf ruby-1.8.7-p72.tar.gz
# cd ruby-1.8.7-p72
# patch -p1 
# export LDFLAGS="-L/usr/sfw/lib -R/usr/sfw/lib -L/opt/local/lib -R/opt/local/lib"
# export CFLAGS="-I/usr/sfw/include -I/opt/local/include"
# export CPPFLAGS=$CFLAGS
# ./configure --prefix=/opt/ruby
# gmake
# gmake install

The above patch basically overrides the iconv that’s used. If you’re feeling fruity, you could grab the source to Sun Web Stack and modify their Ruby, which includes DTrace patches.

Entry Filed under: General