聚會時間公告: 10月份聚會為10月25號星期六下午兩點在MocaMona / 講者:

七月 14, 2008
» [tips] rewrite debian/ubuntu 's lighttpd conf script from perl to python

Today I want to port lighttpd on another platform which basically a debian sarge system but without perl and dpkg package system on it. Since it's a debian based platform so I start from porting debian's binary lighttpd package, however I've found there're some perl script lays in /usr/share/lighttpd which are used when lighttpd startup.

While I can easily dump the result of perl script into a textfile,
and then startup my lighttpd correctly, I thought "maybe port it to python is not a bad idea." (since my target platform has python!), so here is the effort:
create-mime.assign.py

#!/usr/bin/python
#
# This script directly translate from debian's lighttpd perl script:
# create-mime.assign.pl
#
# Author: timchen119.at.nospam.gmail.com
# License: Public Domain
#
import sys

try:
f = open("/etc/mime.types",'r')
extensions = {}
print "mimetype.assign = ("
for line in f:
line = line.strip()
if line.startswith('#'): continue
if line != "":
splitlist = line.split()
if len(splitlist) < 2: continue
mime = splitlist[0]
for ext in splitlist[1:]:
if ext in extensions.keys(): continue
extensions[ext] = 1
print '".%s" => "%s",' % (ext,mime)
f.close()
print ")"
except Exception,e:
print e
sys.exit(1)


include-conf-enabled.py
#!/usr/bin/python
#
# This script directly translate from debian's lighttpd perl script:
# include-conf-enabled.pl
#
# Author: timchen119.at.nospam.gmail.com
# License: Public Domain
#

import os,glob

confdir = "/etc/lighttpd/"
enabled = "conf-enabled/*.conf"

os.chdir(confdir)

for file in sorted(glob.glob(enabled)):
print 'include "%s"' % file

use-ipv6.py
#!/usr/bin/python
#
# This script directly translate from ubuntu's lighttpd perl script:
# use-ipv6.pl
#
# Author: timchen119.at.nospam.gmail.com
# License: Public Domain
#

import socket

##this sometimes not accurate. (like in vserver mode)
#if socket.has_ipv6:
#

try:
if socket.socket(socket.AF_INET6,socket.SOCK_STREAM,0):
print 'server.use-ipv6 = "enable"'
except:
pass

All of these files can be found in http://kalug.linux.org.tw/~tim/lighttpd-debian-python-script/
Well something quite interesting happened when I port the debian's create-mime.assign.pl into python, It's that my python script's final result is not equivalent to perl one and has more mime types than its :
--- perlmime.txt    2008-07-14 15:29:23.000000000 +0800
+++ pymime.txt 2008-07-14 15:29:33.000000000 +0800
@@ -114,6 +114,11 @@
".dvi" => "application/x-dvi",
".rhtml" => "application/x-httpd-eruby",
".flac" => "application/x-flac",
+".pfa" => "application/x-font",
+".pfb" => "application/x-font",
+".gsf" => "application/x-font",
+".pcf" => "application/x-font",
+".pcf.Z" => "application/x-font",
".mm" => "application/x-freemind",
".gnumeric" => "application/x-gnumeric",
".sgf" => "application/x-go-sgf",
@@ -193,6 +198,11 @@
".pk" => "application/x-tex-pk",
".texinfo" => "application/x-texinfo",
".texi" => "application/x-texinfo",
+".~" => "application/x-trash",
+".%" => "application/x-trash",
+".bak" => "application/x-trash",
+".old" => "application/x-trash",
+".sik" => "application/x-trash",
".t" => "application/x-troff",
".tr" => "application/x-troff",
".roff" => "application/x-troff",
@@ -282,6 +292,7 @@
".tgf" => "chemical/x-mdl-tgf",
".mcif" => "chemical/x-mmcif",
".mol2" => "chemical/x-mol2",
+".b" => "chemical/x-molconn-Z",
".gpt" => "chemical/x-mopac-graph",
".mop" => "chemical/x-mopac-input",
".mopcrt" => "chemical/x-mopac-input",

So I start to dig why this happened, and I've found a strange perl regex filter all these mimetypes out, I believe it's a minor bug in original perl program. (or it does implicitly doing something meaningful? well I can't figure it out.)
--- create-mime.assign.pl    2008-07-14 15:35:58.000000000 +0800
+++ create-mime.assign.pl.new 2008-07-14 15:36:07.000000000 +0800
@@ -7,7 +7,7 @@
chomp;
s/\#.*//;
next if /^\w*$/;
- if(/^([a-z0-9\/+-.]+)\s+((?:[a-z0-9.+-]+[ ]?)+)$/) {
+ if(/^([A-Za-z0-9\/+-.~%]+)\s+((?:[A-Za-z0-9.+-~%]+[ ]?)+)$/) {
foreach(split / /, $2) {
# mime.types can have same extension for different
# mime types

replace this line and this will produce same results as mine.

usage:
just copy these py scripts to /usr/share/lighttpd
and change these lines if you're using debian based system
#### external configuration files
## mimetype mapping
#include_shell "/usr/share/lighttpd/create-mime.assign.pl"
include_shell "/usr/share/lighttpd/create-mime.assign.py"

## load enabled configuration files,
## read /etc/lighttpd/conf-available/README first
#include_shell "/usr/share/lighttpd/include-conf-enabled.pl"
include_shell "/usr/share/lighttpd/include-conf-enabled.py"

六月 3, 2008
» [tips] add bzr sftp support when you have no compiler on target platform (pure python )

bzr depends on paramiko to provide sftp support. While paramiko itself is pure python, its dependency pycrypto is not. PyCrypto have lots of C-extenstion and you'll need a compiler to install it. However since we only use part of pycrypto (to have sftp support for bzr), we could just add some stub files to prevent the [deploy] problem.

I have made a modified pure python version pycrypto and packaged it with paramiko 1.7.3, so after you installed bzr (use standard python setup.py or easy_install), you just extract paramiko-1.7.3-bzr-sftp-purepy.tgz
at your python site-package directory (make sure you don't have paramiko and pycrypto already exists, if you do, you don't need to install this package anyway) and happy bzr...!

This also makes bzr only depend on python so you could easily deploy it on a machine which doesn't have c compiler and still have sftp support.

Warning: this pacakage only add bzr sftp support and provides nothing besides this, and these COULD break other python packages which also used paramiko and pycrypto, so don't use it if you don't really need it. And the only tests I've done is on my own (embedded linux) machine, Basically it's just for my own use, I have warned you.

五月 14, 2008
» [tips] 如何讓你的ext2/ext3在神出鬼沒的地雷戰場上存活.

喜歡用自由軟體的人其實應該都滿常遇到地雷,
通常也練就了一身人間即時掃雷機的本事,
但有些時候實在是地雷太小顆 (但是倒炸的很大力),
又發生在想都想不到的地方, 要讓人不嗚呼哀哉也難.
就像開車時你不超車會有別人超車,
你不想用新版會有別人用新版,
軟體相容性的問題往往是會不請自來的.

lloyd大大今天跟我說了一個最近踩到地雷的故事,
他拿了一顆用ext2格式化過的400g硬碟,
拿到他弟弟灌了ext2 driver的windows上執行,
之前好一陣子都能讀取寫入, 操作上都沒問題,
最近卻怎麼格式化都不能用.
(在windows上會問你要不要重新格式化)
換了小一點的硬碟也不行. 最後他深入追查才發現es2fprogs這個最近更新的套件更新了mkfs.ext2這個程式, 預設的inode改變成256 bytes. 所以要用
mkfs.ext2 -I 128 讓預設的inode設成原本的128 bytes.

ok問題解決了, 聽起來只是windows ext2 driver跟e2fsprogs相容性的問題對不對?
但仔細一想問題可能就很大了, 今天你在debian lenny格式化了一顆ext2硬碟, 要放到穩定的重要server上(恰巧是debian sarge),卻不能讀了.
今天如果你沒有"恰巧"讀到這段,


E2fsprogs 1.40.5 (January 27, 2008)

Fix a potential overflow big in e2image if the device name is too long.

Mke2fs will now create new filesystems with 256 byte inodes and the ext_attr feature flag by default.
This allows for much better future compatibity with ext4 and speeds up extended attributes even on ext3 filesystems.

並把他放在心上的話, 你很可能就炸掉了.
(不過事實上可能就算你讀到這段也還是會被炸掉...)

此外/boot通常有人會用ext2而非格式化成xfs或raiser3什麼的(甚至連ext3都不用, 因為穩定),也免不了會踩到這個雷,
這裡"恰巧"就有個血淋淋的例子. (GRUB vs. the Inodes: Who Needs a Bootable System, Anyway? ) 喔, 只是不能開機而已嘛...orz

備註:
e2fsprogs version:
Gentoo-stable: 1.40.8
lenny (next debian stable): 1.40.8
etch (debian stable): 1.39+1.40

重要指令:

mkfs.ext2 -I 128 /dev/???
mkfs.ext3 -I 128 /dev/???

如果你還要向前相容性的話, 從現在開始別忘了mkfs.ext3時加上-I 128 , 否則... 就歡樂的炸吧... XD

感謝lloyd大大更正: 在debian etch (kernel 2.6.18) 上應該還是可以讀取256 bytes inode的格式, sarge是2.4 kernel可能就不行了. (根據mkfs.ext2的man page說法是2.4 kernel會沒辦法mount)

update: fix link.

四月 22, 2008
» [tips] poorman's uudecode

For those who cursed by ash and awk, welcome to 80's !

now you can send everything in ASCII -- hooray!!

#!/bin/sh
#
# uudecode.sh
#
# Author: Ximian, Inc.
#
# Modified: by timchen119 at http://timchen119.blogspot.com from Ximian go-gnome's GPL code
#
# Usage: uudecode.sh encodedfile > decodefile
#
# License: GPL
#
# Download: http://kalug.linux.org.tw/~tim/gpl/uudecode.sh
#
# Ref: http://ftp.cesnet.cz/pub/ximian-gnome/installers/go-gnome
#
# The Ximian Desktop Pre-Installer
#
# Comments to:
# distribution@ximian.com
#
# Copyright 2000-2001, Ximian, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
# General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307,
# USA.
#
#
# This script and its embedded programs are distributed with
# absolutely, positively NO WARRANTY WHATSOEVER, without even the
# implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
# PURPOSE. The author and Ximian, Inc. take no responsibility for
# the consequences of running this script.
#
#

CACHEDIR=/tmp
_awk="awk"

# poor man's uudecode
_awkprog="$CACHEDIR/uudecode.awk"
# encoded file
_uudecode_in=$1

#awk-script
cat > ${_awkprog} <<EOF
function x(l, p) {
n="!\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\\\]^_";
return index(n,substr(l,p+1,1));
}
/^begin/ {}
/^[^be]/ {
len = x(\$0, 0);
for (i=1;len>0;i+=4) {
a=x(\$0,i);b=x(\$0,i+1);c=x(\$0,i+2);d=x(\$0,i+3);
printf("%c",a*4+b/16);
if (len>1) {
printf("%c",b*16+c/4);
if (len>2) {
printf("%c",c*64+d);
}
}
len-=3;
}
}
EOF
${_awk} -f ${_awkprog} < ${_uudecode_in}
rm -f ${_awkprog}


get uudecode.sh at here.

四月 15, 2008
» [tips] upgrading to gentoo 2008.0

Well just as usuals, you can upgrades to gentoo 2008.0 quickly. (beta?)

#eselect profile list
Available profile symlink targets:
[1] default-linux/x86/2006.1
[2] default-linux/x86/no-nptl
[3] default-linux/x86/2006.1/desktop
[4] default-linux/x86/2007.0
[5] default-linux/x86/2007.0/desktop
[6] hardened/x86/2.6
[7] selinux/2007.0/x86
[8] selinux/2007.0/x86/hardened
[9] default/linux/x86/2008.0
[10] default/linux/x86/2008.0/desktop *
[11] default/linux/x86/2008.0/developer
[12] default/linux/x86/2008.0/no-nptl
[13] default/linux/x86/2008.0/server
[14] hardened/linux/x86

For my case, just type 'eselect profile set 10'
and emerge world again.

I'm not migrating to OpenRC and baselayout2 yet, however I'm using KDE4.0.3 now, so I think I have enough un-stable-ness to worry about. ;0

二月 6, 2008
» Am I getting old or ....The lenny is coming this year???

根據 http://lwn.net/Articles/267722/的說法,

下一個版本的 debian (lenny) 將在今年9月release.

這可真是出乎我預期的快... etch... 不是去年四月才出的嗎?

**<小孩勿看>** 大便 難道真的落屎了嗎? **< / 小孩勿看>**

然後話說 /bin/sh link to dash 有了ubuntu帶頭衝之後 還真的debian就跟進了.

我的預言(詛咒)成真?! 看來lloyd大又要怨恨了 :P

(話說 難道再下版inittab也是難逃upstart之手了嗎??? ...有待下回茅房分解...)

這一行果然是很難混的啊.... XD

還有python 2.5.X 啥時要給我進default啊... 時間也該到了吧...

七月 22, 2007