Proiectele din această ediție CDL sunt disponibile pe http://cdl.rosedu.org/2012/proiecte

Au început înscrierile CDL. Proiectele vor fi postate în curând. Va fi fun și interesant :D

N-am fost niciodată student acolo dar de fiecare dată a fost super tare și super awesome. A progresat de la primele ediții, exact așa cum au progresat și WoUSO și celelalte proiecte ROSEdu. Chiar promite a fi ceva fascinant, cool și awesome.

Deci, vizitați link-ul următor, alegeți o problemă de înscriere și înscrieți-vă.

http://cdl.rosedu.org/2012/

(24/6)

Twenty years from now you will be
more disappointed by the things
that you didn’t do than
by the ones you did do.

So throw off the bowlines.
Sail away from the safe harbor.
Catch the trade winds in your sails.

Explore. Dream. Discover.
Mark Twain

Anunțat de multă vreme, ca hint sau direct pe G+/FB/Twitter, acest articol trebuia să apară cel târziu duminică seara dar diverse motive — majoritatea urmând să fie dezvăluite pe parcursul lui — au făcut ca el să ajungă public cu această mică(?) întârziere. Sorry for that.

Ca în fiecare an, articolul de pe 15 ianuarie (sau cu eventualul delay de câteva ore) reprezintă o retrospectivă a lucrurilor din ultimul an precum și un nou New Year Resolution. Așa va fi și acum. Încerc să-l țin cât mai scurt posibil, cel puțin pentru prima impresie.

Fac asta pentru că ziua reprezintă multe lucruri pentru mine, 24-ul din subtitlu nu e la întâmplare iar 6-le reprezintă anul de când m-am apucat de blogging. Plus că între 1 și 15 am timp să analizez trendurile pentru a vedea ce ale NYR-uri își creează oamenii pentru a mă putea ajusta dacă navighez prea mult împotriva furtunii — nu pentru a mă lua după turmă, știți doar că «eu [însă]-n fața normei nu mă-nchin».

Voi face câteva retrospective din mai multe puncte de vedere pentru cel mai încărcat an de până acum și apoi voi enunța câteva planuri pentru anul viitor, unele mai detaliate, altele nu. Împreună cu link-uri către chestii relevante, dacă este cazul.

Dar pentru asta, este nevoie să mergeți la pagina următoare.

next page

On modern Linux distributions, the users have two main possibilities of configuring the network: ifconfig and ip.

The ifconfig tool is part of the net-tools package along side other tools like route, arp and netstat. These are the traditional userspace tools for network configuration, made for older Linux kernels.

The iproute2 is the new package that comes with the ip tool as replacement for the ifconfig, route and arp commands, ss as the new netstat and tc as a new command.

There are pros and cons for each of them and there are users (and fans) of each. Let’s see the differences…

First of all, why was the iproute introduced? There had to have been a need for it… The reason was the introduction of the Netlink API, which is a socket like interface for accessing kernel information about interfaces, address assignments and routes. The tools like ifconfig used the /proc file hierarchy (procfs) for collecting information. The output was reformatted data from different network related files in /proc.

alexj@hathor ~/techblog $ strace -e open ifconfig eth0 2>&1|grep /proc
open("/proc/net/dev", O_RDONLY)         = 6
open("/proc/net/if_inet6", O_RDONLY)    = 6

The costs for the operations like open and read from these files were rather big compared for the netlink interface. For comparison, let’s assume that we have a large number of interfaces (128) with IPv4 and IPv6 addresses and their associated connected routes.

alexj@hathor ~/if $ time ifconfig -a >/dev/null 

real	0m1.528s
user	0m0.080s
sys	0m1.420s

alexj@hathor ~/if $ time ip addr show >/dev/null

real	0m0.016s
user	0m0.000s
sys	0m0.012s

But most of normal users are not that geeky to care about millisecond speedup. They do, however, care about usability. And iproute2 does seem to have a better user interface. The ip command is better organized, in what they called objects. Links, addresses, routes, routing rules, tunnels are all objects, that can be added, deleted or listed. If a user learns how to add an address, by intuition, he can easily guess how to add a route, for example, because the syntax in similar.

Keyword shortening and auto completion makes the ip command more efficient by removing redundant characters. The following commands are identical as effect:

ip address show
ip address
ip addr show
ip a s
ip a

Some network engineers will like iproute2 because it’s similar to Cisco’s IOS: “ip route show” in Linux vs “show ip route” in IOS. Another usability feature is that you have the \number format for subnet masks instead of the quadded-decimal format, the first one being shorter to write and more up to date with the concept of VLSM.

So what does ifconfig still have to keep it around? Its biggest weakness is its biggest strength: its age. ifconfig has been out and used for so long that it’s very hard to put it away. Still many scripts in the heart of Linux distributions rely on ifconfig to work and most system administrators are used to the ifconfig command and it’s hard to move them to something new and unfamiliar. A lot of tutorials on the Internet about network configuration teach ifconfig and not iproute2 to beginners. For example, LPIC-1, one of the biggest Linux Certification out there, still requires ifconfig skills for passing the exam and barely mentiones iproute2.

When released, iproute2 had at least one advantage over ifconfig, and that was the feature of interacting with the IPv6 stack while ifconfig was only for IPv4. But since then, fans of ifconfig patched it so it could also be IPv6 ready.

But other features were not replicated. In old Linux Kernels, an interfaces could have only one IP address, so in ifconfig you could configure only one IP address on an interfaces. In newer kernels, each interface has a list of addresses and iproute2 via the NetLink interface could manage them. Latest ifconfig versions still rely on the idea of subinterfaces to provide more than one address on an interfaces.

So, given all these arguments, iproute2 should be declared the winner. But it’s not that easy. Just like in the case of IPv4 vs IPv6, where the latter one is the obvious choice, iproute2 will eventually replace ifconfig. Only it’s going to take a long time for that to happen, so net-tools will still be around for some time, but they will be eventually phased out.

Each new user of a new system uncovers a new class of bugs.
Kernighan

S-a terminat a 5-a ediție de World of USO anunțat la începutul semestrului prin teaser și teaser extins. Ieri a fost premierea. Ediția din acest an a ieșit cel mai bine de până acum, a avut cel mai mare impact în rândul studenților și a fost jucată de cei mai mulți oameni.

Am un set de statistici, îl voi prelucra și voi posta câteva rezultate interesante mai târziu. Până atunci, clasamentul actual, păstrat pentru eternitate:

După cum observați, avem o competiție strânsă între primele locuri, e primul an în care s-a întâmplat asta. De fapt, e și primul an în care Final Quest nu a fost terminat, primul an în care Final Quest e într-un format nou, primul an în care Final Quest vine și cu Easter-Eggs, primul an în care Final Quest modifică semnificativ clasamentul pentru ultimele poziții din top 10 în loc de primele. E primul an în care avem versiunea în Django, primul an în care avem clase și rase, primul an în care avem bazar, vrăji, conversii de la puncte la bănuți și invers.

A fost un an în care s-au experimentat multe lucruri și mă bucur foarte mult că toate au ieșit ok. Chiar dacă am contribuit extrem de puțin, mult mai puțin decât aș fi vrut.

În final, mulțumiri trebuiesc aduse Eau de Web și grupului Systems pentru ajutorul oferit la premiere, dezvoltatorilor (mexicanii și echipa de conținut) și participanților la joc pentru toată activitatea din timpul semestrului și nu numai.

Mai sunt lucruri de făcut, poate unii dintre studenții de anul 1 sau alți doritori se vor implica în proiect pe partea de dezvoltare sau pe conținut în anii următori. Mai sunt multe idei de implementat, WoUSO promite a crește foarte mult în următoarea perioadă. A crescut extrem de mult de la prima ediție (chiar dacă la început părea că îl consider doar o simplă parodie niciodată nu a fost așa) până acum, fie ca această creștere să fie exponențială :)

The invention of the wheel was perhaps rather obvious;
but the invention of an invisible wheel,
made of nothing but a magnetic field,
was far from obvious,
and that is what we owe to Nikola Tesla.
Reginald Kapp (1956)

Acum câteva săptămâni a fost un curs de CDL interesant. Plănuisem să scriu articolul ăsta atunci doar că timpul a fost împotrivă și aveam multe taskuri de făcut și totul a fost amânat. Not anymore.

Practic, a fost cursul la care Andrei Pitiș – participant la fiecare ediție a CDL-ului – era invitat. Despre prezentarea lui vreau să vorbesc. Ca și prima din serie, a fost una motivațională și foarte interesantă (am lipsit la cea de anul trecut care s-a desfășurat între aceiași parametri).

Ideea e scurtă. Fiecare din cei prezenți în sală – studenți, mentori, invitați, vizitatori – a trebuit să spună cum se vede el peste o vreme, presupunând afirmația «orice trebuie să se îndeplinească ca să ajungi acolo s-a îndeplinit, deja ești acolo unde vrei să fii» adevărată. Citiți și articolul de la prima prezentare, e oarecum legat de ăsta.

Unii au vrut să fie sofware engineer la multinaționale, alții au vrut să fie lideri de mișcare cu impact global, somități, personalități cunoscute, să țină prezentări la TED, să scrie o carte, să contribuie la kernel, să devină un profesor renumit sau de ce nu un Education Evangelist, să fie CEO la un startup de muzică, să contribuie la the next thing, să creeze hardware și software revoluționar, să fie expert în big data handling. O dorință a fost să fie un actor renumit având programarea ca hobby. De aici am aflat că și Florian Pittiș avea acest hobby deși e foarte puțin cunoscut.

După cum vedeți, vise mărețe și diversificate. Fiecare din noi le are și lucrează spre a și le îndeplini mai mult sau mai puțin. Cu trecerea timpului ne adaptăm și/sau renunțăm și ele se modifică. Pe măsură ce avansăm în viață devin din ce în ce mai mici, ajungând la un moment dat la «să am o pensie bună». And that’s just sad. 30 years from now, it won’t matter what jeans you wore, how popular you were, what your hair looked like, or what sneakers you wore. What will matter is what you made of your education, and where it took you in life. Pe măsură ce regretele își fac loc în viață îmbătrânim și vrem din ce în ce mai puțin.

Revenind pe o cale optimistă, în dreptul numelui meu apare «software Tesla, crazed genius». Am tot zis că am multe idei și multe lucruri de făcut și că trebuie doar timp și voință și dedicare să mă apuc de ele. Chiar dacă par ciudate când le povestesc prima dată, tot aș vrea să le implementez, să văd ce iese. Unele ar putea fi ahead of time dar atunci apar doar sugerate prin diverse articole pe aici, diverse teorii. Oricum, calea asta, 42, e strâmtă și uneori e nevoie de mai mult de unul singur pentru a o parcurge.

Ca să termin, revin la un citat din BSG. «I want to see gamma rays! I want to hear X-rays! And I want to – I want to smell dark matter! [...] I know I want to reach out with something other than these prehensile paws! And feel the wind of a supernova flowing over me!». Până atunci mai este mult dar viitorul se face azi. Să vedem ce voi zice la următoare ediție CDL.

Stack space is the part of each process’ virtual memory where function arguments and return addresses are stored, along with local variables declared within a function. Usually, the stack begins at the high address space of the virtual memory and grows down.

At every function call, a new stack frame is created on the stack. It contains the parameters sent to the function, the return address (the address of a code in the caller function) and the locally declared variables.

For each function call, the SP/ESP (Stack Pointer/Extended Stack Pointer) is set so the stack has a big enough size to accommodate local variables. For example, in theory, if you have a local char variable and an int variable, the SP should be set (moved) to 5 bytes.

In practice, the compiler will allocate stack space a little different than expected. It will allocate local variables space in increments of a fixed size, so sometimes having two int variables or three int variables will be the same.

As an example, gcc will allocate in increments of 16 bytes. Let’s make an experiment… we take a simple C program and turn into assembly code.

The C file looks something like this:

int main(void)
{
	int a=1, b=2;
	return 0;
}

The variables must be used after declaration or they will be ignored by the compiler.

The resulting assembly code (with an gcc -S) looks like this:

main:
	pushl	%ebp
	movl	%esp, %ebp
	subl	$16, %esp
	movl	$1, -4(%ebp)
	movl	$2, -8(%ebp)
	movl	$0, %eax
	leave
	ret

Notice the subl instruction that clears 16 bytes in the stack space by decrementing the ESP. Those 16 bytes are enough for four 32bit integers. If you have 1,2,3 or 4 local variables declared (and used), you get those 16 bytes.

If we declare 5 integers, the allocated space will now be 32bytes. Same thing for 6, 7, or 8. If we have 9 to 12 integers the compiler will allocate 48 bytes. An so on…

What if we don’t only have integers? Let’s add some chars.

int main(void)
{
	int a=1, b=2;
	char c=3, d=4;
}

Result:

main:
	pushl	%ebp
	movl	%esp, %ebp
	subl	$16, %esp
	movl	$1, -8(%ebp)
	movl	$2, -12(%ebp)
	movb	$3, -1(%ebp)
	movb	$4, -2(%ebp)
	movl	$0, %eax
	leave
	ret

The function would need 10 bytes, but still gets 16. So the allocation is in increments of 16 bytes no matter what.

The question remains why? It has to do with the cache alignment. The compiler will try to structure the memory usage so that the executed code can be easily fetched from memory and cached. A correct alignment will cause minimum cache misses for memory access.

Credits to SofiaN for help with initial observations and tests.

Anul acesta intenționăm să continuăm Inițiativa de Crăciun și sperăm, cu ajutorul celor care vor să se implice, să realizăm ceva cel puțin la fel de frumos ca în anii trecuți.

Pe scurt, pentru cei care nu au avut ocazia anii trecuți, inițiativa noastră constă în colectarea de cadouri: haine, cărți, rechizite, orice este la îndemână și ar putea folosi unor copii care nu au posibilitatea unei vieți decente.

Colectarea are loc până în ultima săptămână de școală (23 decembrie 2011) în ED422 și în EG106 (Laborator IXIA), Facultatea de Automatică și Calculatoare, UPB.

Pentru mai multe informații, puteți folosi site-ul inițiativei[1].

[1] http://donatii.cs.pub.ro/

Vă mulțumim!

Haideți să-i ajutăm și noi :)

As we all know, IPv6 is the new protocol of the Internet, that will come to replace the current version of IP (Internet Protocol), IPv4. It will come to fix the flaw of the 32 bit addressing in IPv4, flaw that led to the current shortage of usable address in the Internet.

The addressing issue is not something new. The IETF started looking into a replacement for IPv4 since 1992-1993, when they started the IPng (IP next generation) discussion group and by 1996, they had the specifications for IPv6.

So considering that the Internet is about 40 years old and the IPv4 addressing problem is been known for about half that time, why is it that after 15 years since having the solution in the form of IPv6, why is it still not predominately used?

Probably the easiest way to have build the IPng is with a backwards compatibility (for example, using a variable length address, like OSI’s CLNP, where all the IPv4’s address space is just a part of the IPv6 space, using 32 bits). But since they wanted to start from scratch an rewrite everything in order to fix other problems in IPv4 (like the now almost useless header checksum) and to add new features (like the header extensions that allows protocols like IPSec to be built inside IPv6). But the “rewrite everything” approach meant that almost all of the components of the network layers had to be rewritten and this resulted in a large groups of people being affected by the change.

First were the network administrators, the ones that had to ensure that their routers, multilayer switches, firewall and wireless controllers were ready to be migrated. Most of the old equipment had to be replaced with new ones, or at least have their software updated. Current equipment do most of their packet processing in hardware to get better performance, but this is valid only for IPv4 packets. Hardware processing for IPv6 packets is something that only very new models of routers and switches do, and companies don’t really want to buy new equipment since the costs are rather big. Routing protocols had to be rewritten or modified or written from zero. OSPFv3, the link state protocol and the simple and lightweight distance vector, RIPng, had to be implemented from scratch. More modular, IP independent protocols like EIGRP and Intergrated IS-IS needed new modules for the new protocol.

System administrators had the same concern, getting their services IPv6 ready. From setting up their web services to listen on both protocols to the more difficult service, DNS. If DNS in IPv4 was a good thing to have, in IPv6, DNS is critical (nobody wants to remember a 32 hexadecimal digit number). The DNS protocol needed to add a new record, the AAAA record, and needed to implement a new reverse DNS zone, the ip6.arpa. zone.

But some of the frustrations of the administrators and the users are caused by bugs or even lack of implementation in software. Since every hardware needs a software, IPv6 first of all needs support in the software written. Kernel, system and application programmers needed starting building in support for IPv6. For example, people started patching Linux 2.1 back in 1996, but real stable, built-in support for IPv6 only came out in 2.6. Support in kernel still didn’t mean that people could use it because it lacked the userspace tools. The wide used ifconfig wasn’t build for v6, and only with the development of iproute2, Linux users could configure IPv6 on their boxes. Although considered deprecated, newer versions of ifconfig do support IPv6 address assignments. In the Windows world, things are worse, since only Windows 7 really has full support (kernel and user space tools) for IPv6.

Only after the IPv6 stack is built inside the kernel (the network stack being one of the hardest part of the kernel to program), the system programmers could start porting their programs to be IPv6 ready. IPv4 and IPv6 sockets are not compatible, because the second one needs to implement the address family AF_INET6. An IPv6 ready application also needs to be IPv4 working, so it needs to be smart and know when to create a v4 connection or a v6 connection. Because if only sometimes a v6 infrastructure is available, a v4 infrastructure is almost sure there. But if both are available, which one do you chose, because maybe one works better than the other in that situation?

So as we can see, there is not one group affected by the migration to IPv6, but rather an entire ecosystem, with several groups affecting each other.

Suppose your favorite application or a library you are using has a bug. You find that the code is open source and are happy because of this. Being a programmer yourself, you know that you can fix the bug and send a patch with the fix to the maintainers. But how do you do this? This article will provide a short walkthrough for this task using as an example the Linux kernel. Different projects use different source version control systems. Because this article works on the kernel tree, I am going to use git as an example.

So, the first thing to do is to clone the project’s repository. This is to ensure that you are working on the latest source – maybe the bug was fixed before and your operating system’s package manager is behind on updates. For our kernel example, we will be cloning the net-next tree since this is where our final patch will land – from there it would be applied to the Linux kernel itself but this process is not the subject of this article.

$ git clone
git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next.git
Cloning into 'net-next'...
remote: Counting objects: 2241130, done.
remote: Compressing objects: 100% (350702/350702), done.
remote: Total 2241130 (delta 1873243), reused 2236736 (delta 1869251)
Receiving objects: 100% (2241130/2241130), 442.07 MiB | 947 KiB/s,
done.
Resolving deltas: 100% (1873243/1873243), done.

Next, create a new branch on which to work and checkout it. All our work will be done there and we will use this branch later when constructing the patch to be sent upstream.

$ cd net-next/
$ git branch speedup_proc_net_dev
$ git checkout speedup_proc_net_dev
Switched to branch 'speedup_proc_net_dev'

Now, do your work, change the source, fix the bug or develop the improvement. Be sure to follow the coding standards of the project you are contributing to. Commit often as told in the git article. At the end of the task, when everything is solved and you are ready to submit the patch, you can rebase the commits into a single one or a set of commits depending on their content – it is better to have a single logical change per commit, also your patch will have an increased chance of being accepted if each commit is small. When rebasing your commits be sure to have a relevant commit message (as per git article for example). For the Linux kernel there is a standard even in the commit message. Start with a single line detailing the component you’re patching and a short description of the commit then – after an empty line – write a longer message detailing what you have done. Add relevant information about the problem that you solved, and – if possible – tests made when developing your solution. Also add a Signed-off-by line. For example, the following is an example of a good commit message.

    dev: use name hash for dev_seq_ops

    Instead of using the dev->next chain and trying to resync at each call to
    dev_seq_start, use the name hash, keeping the bucket and the offset in
    seq->private field.

    Tests revealed the following results for ifconfig > /dev/null
	* 1000 interfaces:
		* 0.114s without patch
		* 0.089s with patch
	* 3000 interfaces:
		* 0.489s without patch
		* 0.110s with patch
	* 5000 interfaces:
		* 1.363s without patch
		* 0.250s with patch
	* 128000 interfaces (other setup):
		* ~100s without patch
		* ~30s with patch

    Signed-off-by: Mihai Maruseac <mmaruseac@ixiacom.com>

Next step is to create the patch files. We do this by switching to the master branch and doing a git format-patch operation.

$ git checkout master 
Switched to branch 'master'
$ git format-patch master..speedup_proc_net_dev 
0001-Speedup-proc-net-dev-filling.patch

As you see, in our case a single file was created since our speedup_proc_net_dev branch was only a commit ahead of the master branch (we previously rebased everything into a single commit). This will be the file containing our patch, the file we will send upstream. But, before going there we still have a lot of things to do.

First of all, we will need to check our patch for coding style mistakes. In the case of the Linux kernel there is a script doing that and we will use it. For other projects, we may need to do this step manually.

$ ./scripts/checkpatch.pl 0001-Speedup-proc-net-dev-filling.patch 
total: 0 errors, 0 warnings, 122 lines checked

0001-Speedup-proc-net-dev-filling.patch has no obvious style problems and is ready for submission.

If there are problems we will have to go back to our branch, fix them, rebase all commits and recreate the patches with git format-patch. When everything is ready to be submitted we can send the patch to the developers via an email. In most projects you will simply create a bug report and attach the fix there and you are done. But since the Linux kernel is more complex we will have to use the email path presented in the following paragraphs.

First of all, we have to find where to send the patch. We have another script which can be used.

$ ./scripts/get_maintainer.pl 0001-Speedup-proc-net-dev-filling.patch
"David S. Miller" <davem@davemloft.net> (maintainer:NETWORKING [GENERAL],commit_signer:118/147=80%)
Eric Dumazet <eric.dumazet@gmail.com> (commit_signer:32/147=22%)
"Michał Mirosław" <mirq-linux@rere.qmqm.pl> (commit_signer:21/147=14%)
Jiri Pirko <jpirko@redhat.com> (commit_signer:15/147=10%)
Ben Hutchings <bhutchings@solarflare.com> (commit_signer:9/147=6%)
netdev@vger.kernel.org (open list:NETWORKING [GENERAL])
linux-kernel@vger.kernel.org (open list)

The addresses given as output are those where we will send our email. But, before sending the first email, we will have to configure git send-email. For example, adding the following lines to ~/.gitconfig will ensure that you can use Gmail as a SMTP server for sending the patch email.

[sendemail]
	smtpencryption = tls
	smtpserver = smtp.gmail.com
	smtpuser = yourname@gmail.com
	smtpserverport = 587

Now, we can send the email. We will have to manually fill in the --to and --cc options or we can use a list of sed commands as suggested by the Chromium wiki. In our case we will do it manually just to exemplify all steps, in real life it will be better to use scripts whenever it is possible.

git send-email --to=netdev@vger.kernel.org \
> --cc=linux-kernel@vger.kernel.org \
> --cc=... 0001-Speedup-proc-net-dev-filling.patch
0001-Speedup-proc-net-dev-filling.patch
Who should the emails appear to be from? [Mihai Maruseac <mihai.maruseac@rosedu.org>] 
Emails will be sent from: Mihai Maruseac <mihai.maruseac@rosedu.org>
Message-ID to be used as In-Reply-To for the first email?
....

After several more lines of output your mail will be sent. I have responded with the default entries to the above questions but the last one is very relevant, as we will see next.

After the mail is sent, it will appear on patchwork and on the mailing lists. You will wait until someone looks through your mail and analyzes your patch. Then, the patch can be applied or someone can report some problems to you. If there are some problems, you will go back and solve them and will resend the patch using the above methodology. This time, you will answer the Message-ID question with the ID taken from the first email. In our case, the patch was not accepted from the start and we had to reiterate. Thus, we answered that question with the ID taken from the initial patch: <1318412950-22014-1-git-send-email-mmaruseac@ixiacom.com>. Until the final patch was accepted I needed to send several versions.

Even though this lasted a whole week, the feeling I got when it was finally accepted was awesome. You will feel it too after sending the first few patches.

As a recommended link before the end of the article, make sure you listen the YouTube video of Greg KH about contributing upstream.

A process is an instance of a binary executable file. This means that when you ‘run’ a binary, the code from the storage media is copied into the system’s memory, more precisely, into the process’ virtual memory space. From a single binary, several processes can be spawned.

The virtual memory of a process, made up of pages, is mapped to several things, like shared objects(libraries), shared memory, stack and heap space, read-only space and executable space. A good way to view what is mapped to what is with the pmap utility, or by just looking in the /proc directory hierarchy. The /proc/$PID/maps file (where $PID is the process ID of the targeted process) has the page mappings. Also in /proc/$PID, you can find other useful files, like the exe file that contains a symlink to the executable or the fd directory that contains symlinks to all the files opened as file descriptors in a process.

Except useful information, what can we get out of the procfs? Here is a situation that has been known to happen. You are in a console, with your bash shell, and you manage to delete some important files, like /bin/bash. Without that executable, you cannot run new shells and on a restart, your system will be inaccessible. What can you do?

The code of your bash is no longer on the hard drive, but it is in the virtual memory of the process you are currently running. You can find out what’s the PID of the current shell instance using $$ enviroment variable . Knowing that, you can cd to the /proc/$$ and access the content of the exe file there.

Although the exe file is shown as a link to the original file that is now deleted (thus the link should be broken), if you cat it, you will get its binary content. In fact, all the original binary file. Here is the step by step process:

/bin # md5sum bash
e116963c760727bf9067e1cb96bbf7d3  bash
/bin # rm bash
/bin # echo $$
5051
/bin # cd /proc/$$
/proc/5051 # ls -la exe
lrwxrwxrwx 1 root root 0 2011-11-15 23:47 exe -> /bin/bash (deleted)
/proc/5051 # cat maps
[snip]
00f9e000-00f9f000 rw-p 0001c000 08:01 263123     /lib/i386-linux-gnu/ld-2.13.so
08048000-0810c000 r-xp 00000000 08:01 284760     /bin/bash (deleted)
0810c000-0810d000 r--p 000c3000 08:01 284760     /bin/bash (deleted)
0810d000-08112000 rw-p 000c4000 08:01 284760     /bin/bash (deleted)
[snip]

/proc/5051 # cat exe>/bin/bash_rescued
/proc/5051 # cd -
/bin # md5sum bash_rescued
e116963c760727bf9067e1cb96bbf7d3  bash_rescued
/bin # chmod +x bash_rescured
/bin # mv bash_rescured bash

What other things can we rescue? How about a file that was opened by a process? For example, a video file, opened by a player:

alexj@hathor ~ $ md5sum movie.ogv
9f701e645fd55e1ae8d35b7671002881  movie.ogv
alexj@hathor ~ $ vlc movie.ogv &
[1] 6487
alexj@hathor ~ $ cd /proc/6487/fd
alexj@hathor /proc/6487/fd $ ls -la |grep movie
lr-x------ 1 alexj alexj 64 2011-11-16 00:11 23 -> /home/alexj/movie.ogv
alexj@hathor /proc/6487/fd $ rm /home/alexj/movie.ogv
alexj@hathor /proc/6487/fd $ ls -la |grep movie
lr-x------ 1 alexj alexj 64 2011-11-16 00:11 23 -> /home/alexj/movie.ogv (deleted)
alexj@hathor /proc/6487/fd $ cp 23 /home/alexj/movie_rescued.ogv
alexj@hathor /proc/6487/fd $ md5sum /home/alexj/movie_rescued.ogv
9f701e645fd55e1ae8d35b7671002881  /home/alexj/movie_rescued.ogv

These things are possible because the instances of the files are still kept and used by the kernel. The VFS (the Virtual File System) still has references to the inodes of the files. They won’t be released until the processes will be finished.

Thanks to razvand and ddvlad for the idea of this article.

In theory, theory and practice are the same. In practice, they’re not.

Nu planificasem deloc să revin la acest articol. Inițial ar fi trebuit să apară în seria inițială a drumurilor dar aia a scăpat de sub control și apoi a fost transformată. Așa că, dacă nu ar fi fost istoria Dianei, nu ar fi apărut acest articol. Îl scriu acum, n-o să-l recitesc, dacă reușesc să ajung până la final îl public, altfel va fi șters (vreau să experimentez să văd ce-ar ieși dacă fac așa).

Primul contact cu un PC l-am avut puțin înainte de școală. La mama la servici. Jocuri. Prince of Persia. PacMan, Supaplex, și multe altele. Inclusiv Heretic, DOOM și Warcraft II. Pentru ultimele 3 chiar îmi făceam program pe acolo doar ca să le pot juca. Așa am ajuns până în a 5-a când am făcut rost de cheat-codes și nu a mai fost la fel de interesant.

Tot în a 5-a am fost la Palatul Copiilor sau cum se chema la cursurile de informatică de acolo. Unde nu am făcut altceva decât `tastatura` pe durata unui întreg semestru. Erau HC-uri de alea pe care fiecare tastă avea mai multe moduri (memoria de atunci nu mai e la fel de perfectă, e posibil să scriu prostii din când în când — sorry) și aia care ar fi trebuit să se ocupe să ne învețe câte ceva era destul de leneșă ca să facă altceva. Așa că mai mult ne adunam în jurul celor 2 Pentiumuri uitate pe acolo să vedem cum se jucau norocoșii din săptămâna respectivă (trași la sorți sau pe rând).

Până când într-o zi n-a putut veni ea și a venit altcineva. Ne-a scris pe tablă un program BASIC, linie cu linie și a stat de noi până l-am trecut și noi pe calculatoare. Și apoi am stat să ne jucăm. Era un hangman simplu în care toate cuvintele erau în codul programului și se mergea circular prin ele. Pregătisem câteva întrebări pentru data viitoare în timp ce mergeam acasă: cum fac să dea cuvintele aleator nu în ordine, cum schimb cuvintele fără să rescriu tot programul, chestii de astea. Nu am mai fost la ăla după săptămâna aia, mama a decis că pierdeam prea mult timp doar ca să învăț tastatura. Aparent, pe la jumătatea anului sau așa ceva s-a schimbat profesorul de acolo și cei care au rămas au început să fie introduși în programare. În Pascal parcă, nu mai știu. Am aflat asta prin a 7-a, pe vremea olimpiadelor. Deja cam târziu.

În generală am făcut ceva informatica în a 5-a: chestii generale cu ENIAC, abac, Babbage, etc terminate cu diagrame logice și ceva pseudocod. Probabil dacă am fi făcut în a 6-a și după ar fi fost ceva. Dar până în a 8-a n-am mai făcut nimic. În a 8-a ne-a îmbuibat mai mult cu suita Office și atât.

Așadar, până în liceu, nu prea am avut contact cu programarea. Exceptând cărțile de acasă pe care le-am citit într-o vreme când citeam tot ce prindeam, după ce am terminat tot ce-am găsit de Jules Verne. Exista o carte introductivă, nelegată de vreun limbaj anume. Mai era una bazată pe COBOL și de atunci nu mi-a plăcut limbajul ăsta. Mai era una de BASIC și una care prezenta mai multe limbaje. Asta ultima avea și gramatici BNF și puțină teorie de limbaje formale. Dar a fost primul contact cu limbaje precum PL/1, LISP și APL. Oricum, pe vremea aia erau doar așa cunoștințe de știut și cam atât, nu aveam pe ce să le testez.

Pentru că abia în liceu am reînceput programarea, pe C și pe primul PC personal. Până atunci am avut timp să acumulez și să înceapă să-mi placă destul de mult matematica și fizica și, într-o oarecare măsură, toate științele exacte. Deja aveam acces la informații despre premiile Nobel, multe povești despre istorie, etc. Dar divaghez foarte mult.

Introducerea în C din a 9-a nu a mers chiar cum m-aș fi așteptat. Și mai aveam de recuperat diferența față de cei care au făcut 4 ani mai mult programare. Plus că aveam nevoie de câte un motiv ca să mai stau la calculator — când nu era nimeni care să supravegheze mai dădeam câte un Alt-Tab în NFS sau în Starcraft (chiar și Diablo la un moment dat până am dat de Heroes 3 care a rămas și acum un joc de suflet). Așa că m-am apucat să implementez și să rezolv singur problemele din Tudor Sorin. Eventual, îmi mai cream și altele. Nu înțelegeam de ce soluția pe care am dat-o pentru un program care să calculeze strategia optimă de a termina un joc de whist cu 0 dura atât de mult dar tot am încercat, de exemplu.

Pe CD-ul cu Borland C mai primisem și un exemplar de Delphi și unul de C++ Builder. Și o versiune de Macromedia Flash. În timpul liceului m-am jucat cu toate și am creat diverse mici aplicații pentru diverse scopuri. Așa de formă, nimic util. Până prin a 10-a, începutul clasei a 11-a când Andrei mi-a propus să implementăm un anumit joc. Pe care am tot încercat să-l implementez până în facultate. Mai am și acum arhivele cu ultima versiune deși nu cred că aș mai merge pe aceeași cale cu care am început atunci. Joc care a contribuit la implicarea mea în Hammerfall – picat acum – și contactul cu Python. Urmat apoi de multe discuții cu Andrei pe diverse teme, idei de jocuri de implementat, idei filosofice, chestii tehnice…

Revenind, un orizont nou s-a deschis în a 10-a odată cu conectarea la interwebs. Multe resurse au fost descoperite. NeHe, AIDepot, Wikipedia, etc. Însetat de cunoaștere am început să acumulez detalii care nu păreau utile atunci. Deja aflasem de Godel și de Peano înainte de facultate.

Dar abia în facultate au început să se lege lucrurile. După ce am făcut cunoștință cu Vlad și cu Haskell. După ce am fost introdus în lumea programării funcționale și după cursul de PP. Încă nu sunt legate complet, se mai leagă și acum lucruri între ele. Mai ales pe durata zilei de joi cu un TSFP și un CCS. Dar și din lecturi suplimentare, din ebook-uri descărcate, din căutări, etc.

Practic, pe măsură ce trece timpul cantitatea de cunoștințe noi e din ce în ce mai mare și din ce în ce mai multe paralele se întrezăresc. Totuși, timpul trece și lucruri de aflat si de descoperit sunt multe..

Second teaser

Mândru locuitor al Isoropiei,

Întruchipezi abilitățile cele mai de preț al castei tale și ai fost desemnat să participi la Jocurile Fonice și să câștigi onoarea de a restaura pe Fonix, Spiritul Focului. Tribul și poporul tău îți admiră calitățile și te susțin frenetic pentru a căpăta locul de cinste pentru reîncarnarea lui Fonix.

Ești dornic și pregătit să participi la aventurile și încercările din cadrul Jocurilor Fonice. Știi că cei mai buni zotieni, oxynieni și nifesoni participă la acest joc. Faptul că cei mai buni vor fi concurenții tăi te animă și te provoacă. Ai suportul castei tale și ai încredere că, prin forțe proprii și cu suportul lor, vei fi unul dintre cei 10 care vor încarna Spiritul Focului. În sinea ta, te gândești cu ambiție la onoarea supremă de a fi întruparea lui Fonix!

Jocurile Fonice au început! Pornește aventura, vajnic luptător!

In omnia paratus

O dupa-amiază fierbinte venea în continuarea unei zile calduroase. Aerul parea că stă în același loc de săptămâni întregi, deși cu doar câteva zile înainte ploile măturaseră întreg ținutul. Rareori se auzeau sunetele unei păsări sau ale unui animal rătacit în căutarea hranei. În zare se putea distinge silueta distorsionată a unei corăbii ce trecea lin peste un pâlc de pădure. Încărcată cu minereu, corabia se îndrepta agale către munții semeți ce răsar la marginea ținutului.

Sub soarele dogoritor patru siluete se apropie de o răscruce de drumuri. Venind din cele patru zări, siluetele par că plutesc peste iarba gălbuie a preeriei.

- Știți de ce v-am invocat. Se apropie momentul! spune, cu glas hotărât, una din siluete.
- Cât timp a trecut? întrebă cea de lângă ea.
- 78 de treceri ale soarelui! se auzi a treia siluetă, cu o voce blândă.
- Deja? Trebuie să dăm de veste muritorilor. Voi merge la Consiliul Academiilor, spuse a patra siluetă.
- Eu merg la castelul celor 5 ghilde, se auzi vocea blândă.
- Iar eu voi ajunge la Adunarea Caselor, răspuse prima siluetă.

Cele patru siluete apucară fiecare pe drumul ei lăsând în urma lor un soare dogoritor ce învăluia totul cu caldura lui.

Momentan doar teaser. În mai puțin de 12 ore va veni și completarea. Pentru ceva mult mult mai fun decât ce-a fost în anii trecuți. Și cu mult mult mai multe variante de joc ;)

The idea for this post came from Virgil’s comment on char[] versus char* entry. We will dig into some of C’s extern keyword internals by means of examples and then analyze the differences between extern char* and extern char[].

extern is a storage class specifier, indicating that the actual storage of a variable or the definition of a function is located elsewhere, typically in another source file.

Let’s start with a simple example:

helper.c

int sample = 42; /* definition */

main.c

extern int sample; /* declaration */
int main(void)
{
	printf("sample = %d\n", sample);
}

Having obtained the corresponding object files helper.o and main.o we link them together into an executable named main. We will use the nm tool to check the symbols from each object file:

$ nm helper.o 
00000000 D sample
$ nm main.o
         U sample
00000000 T main

Notice that the symbol sample is only declared in main.c but not defined there. In the linking phase, the linker searches throughout all linked object files and finds out that the actual storage for sample is defined in helper.c. As a result our main executable will print value 42 declared in helper.c external file:

$./main 
sample = 42

Now let’s see how the compiler behaves if the types for cross-referenced variables do not match:

foo.c

char *foo = "Hello";

main.c

void foo(void);

int main(void)
{
	foo();
	return 0;
}

$ gcc -Wall -c foo.c -o foo.o
$ gcc -Wall -c main.c -o main.o
$ gcc -o main main.o foo.o
$ ./main
Segmentation fault

Functions are by default extern, hence the declaration of symbol foo in main.c file allows the compiler to create main.o object file without errors or warnings. Anyhow, the linker does not check the type of symbol foo; thus, running the main executable results in a function call into an non-executable memory area.

Finally, let’s analyze if we can use a pointer and an array interchangeably between 2 source files.

First try. The file main.c declares an extern array of chars, leaving it to the linker to find the actual storage area defined for it. File pointer.c defines a pointer to a memory area holding a string literal. At link time, the symbol str from main.c is bound to a memory area representing the address of a string.

pointer.c

char *str = "1234";
char a = 'A'; /* memory guards */
char b = 'B';
char c = 'C';

main.c

extern char str[];

int main(void)
{
	printf("%s\n", str);
	return 0;
}

By compiling and linking main.c and pointer.c together we get main executable.

$ ./main
\�ABC

Second try. The file main.c declares a pointer to a memory area holding one or more characters. The linker will associate str from main.o with the storage defined by str array from array.o.

array.c

char str[] = "1234";

main.c

extern char *str;

int main(void)
{
	printf("%s\n", str);

	return 0;
}

By compiling and linking together these programs we notice that running the main executable results in a crash.

$ ./main
Segmentation fault

$gdb ./main
(gdb) b main
Breakpoint 1 at 0x8048385: file main2.c, line 6.
(gdb) run
Breakpoint 1, main () at main2.c:6
6		printf("%s\n", str);
(gdb) p str
$1 = 0x34333231 Address 0x34333231 out of bounds

One can notice that the value of the pointer str is the content of array str. This content is an invalid address dereferenced by the pointer, resulting in the delivery of the dreaded SIGSEGV signal.

This article aims to shed some light on the topic of library management with insight on the linker and loader. The ldconfig command, for example, is heavily used in Linux, though unknown to some of users.

A library is a collection of object files “meshed” together in another file. Its benefit is avoiding “reimplementing the wheel”. Once one has implemented a given set of functionalities, he/she may store those in a library file; this file is distributed to others and used in various software projects. Libraries are heavily used in all modern operating systems; the greater part of packages in Linux distributions are library packages. One can barely imagine being able to do any kind of development without the presence of the C Standard Library on the local system.

Linking and Loading

A library is said to be “linked” together with other library files or object files into an executable. The executable integrates all required components from library files, avoiding the need of implementing these components from scratch.

Linking is thus the process where external references in each module (object file) are resolved; that is, undefined functions are now looked in other linked modules or library files and their code is used in the executable. The linker is the application responsible for resolving and integrating functions in the end executable file.

With respect to the phase when linking occurs, we differentiate between three types of linking:

1. static linking
2. load-time dynamic linking
3. run-time dynamic linking

The above nomenclature is specific to MSDN (load-time dynamic linking and run-time dynamic linking) but it’s a good depiction of any system using dynamic linking.

When using static linking, required library function code is inserted into the executable at link-time. Link-time refers to the moment when the linker process (ld) is invoked (typically wrapped by the gcc command). The result is an executable that comprises all required code to create a process.

When using dynamic linking, the linker process does not integrate code from the library. It simply creates stubs in the executable code stating what library file should be looked for that function. The actual “linking”, that is the “integration” of code in the executable, is done later.

Depending on the “later” part of dynamic linking, we differentiate between two types of linking. Load-time is when a process is created from an executable; the loader is responsible for “transforming” an executable into a process (actually, it’s not a transformation, but an instantiation). Run-time is the time while the process is running (using memory space, running code on the CPU etc.).

For load-time dynamic linking, the linking is done at load-time. That is, when running the executable (./myexec) and when the process is created, code from the library is mapped into memory and then referred to by the newly created process. For run-time dynamic linking, a specialized API allows the developer to load the library code into memory and, on demand, use specific functions.

Library types

Modern OSes such as Windows, Linux, Mac OS X and other Unices use two types of libraries, strongly related to the types of linking shown above: static libraries and dynamic libraries. Static libraries are used in conjunction with static linking, while dynamic libraries with load-time/run-time dynamic linking.

Static libraries use the .a extension on Unix and .lib on Windows. Each time some modules are linked against a library file, static linking is enabled and code for functions used is copied into the executable file.

    ar rc libtest.a module1.o module2.o
    gcc -o myexec exec.o -L. -l test

Dynamic libraries are called shared-object library on Unix and use the .so extension on Unix. On Windows, they are called dynamic-link libraries and use the .dll extensions. If a shared-object library is linked against a module, only references to the library are filled, no actual code is copied; that step is done later on (either at load-time or run-time).

In order to use a shared-object library for load-time linking, one would simply pass it as an argument to the linker:

    gcc -share -fPIC -o libtest.so module1.o module2.o
    gcc -o myexec exec.o -L. -l test
    LD_LIBRARY_PATH=. ./myexec

When the loader creates a new process (LD_LIBRARY_PATH=. ./myexec), the library (libtest.so) is mapped into memory and necessary function code is accessed.

The use of run-time linking requires a specialized API for loading needed function code while the process is running: dlopen & friends. A sample is shown below:

    double (*cosine)(double);

    handle = dlopen ("libm.so", RTLD_LAZY);
    cosine = dlsym(handle, "cos");
    printf ("%f\n", (*cosine)(2.0));

Unlike static and load-time dynamic linking, run-time dynamic linking doesn’t require the presence of a library argument to the link command (that is -L. -ltest).

Advantages of a certain type of library (static or dynamic) are disadvantages for the other one and vice versa.

Static library-generated executables have increased portability. All code is inserted into the executable such that, moving it on a different platform doesn’t require the presence of that library. These executables tend to be faster as no additional overhead is implied during load-time or run-time.

Dynamic library-generated executables have two main advantages: they are smaller in size and library files have a smaller memory footprint. The first advantage is due to not copying function code at link time: only references are added to the executable without additional code. The second advantage is stated in the Unix name for dynamic libraries: shared-object libraries. A library may be mapped in memory and all processes that use the library would use the same code. Thus, 50 processes that use the C standard library would require a single instance of the library to be mapped in memory.

Library Management

When discussing about library management, we are talking about dynamic libraries. This is due to the fact that, when using the library code (either at load-time or run-time), the loader needs to know where to find the requested libraries.

The Linux loader is called ld-linux.so. As stated in the man page: “The programs ld.so and ld-linux.so find and load the shared libraries needed by a program, prepare the program to run, and then run it.” The loader needs to lookup shared libraries in order to run the program and instantiate a process.

Bear in mind that the -L. option passed to GCC when doing linking is only used at link-time. It’s used to locate the library at link-time, not at load-time or run-time.

In order to configure the loader to lookup libraries for dynamic linking in a given folder (for example, the current folder – .), there are two main options: using the LD_LIBRARY_PATH environment variable or the ldconfig command.

The LD_LIBRARY_PATH variable is a list of colon delimited folders where libraries are searched. It must be set when the loader is invoked – that is, when running the executable:

    export LD_LIBRARY_PATH=.
    ./myexec

Using the LD_LIBRARY_PATH variable is excellent for testing. It does however pose two disadvantages: it does not allow persistent configuration and it may suffer from security vulnerabilities similar to the PATH environment variable.

The configuration approach is the use of the ldconfig command. ldconfig is used to populate the library list cache file /etc/ld.so.cache. The cache file is read by the loader to search for libraries. On Debian-based systems, every time you install a library, ldconfig is run to populate the cache file.

In order to incorporate a new folder in the library search path, one may resort to a persistent configuration or a temporary one. For a temporary run, simply pass the new folder to ldconfig:

    razvan@einherjar:~/code$ /sbin/ldconfig -p | grep libtest
    razvan@einherjar:~/code$ sudo /sbin/ldconfig /home/razvan/code/
    razvan@einherjar:~/code$ /sbin/ldconfig -p | grep libtest
    	libtest.so (libc6,x86-64) => /home/razvan/code/libtest.so

For a persistent, configuration, one would need to edit the configuration file and/or folder for ldconfig, namely /etc/ld.so.conf and /etc/ld.so.conf.d/. Simply add a new folder in the configuration file and run ldconfig.

When using dlopen & friends, the same kind of configurations may be used: LD_LIBRARY_PATH, temporary use of ldconfig and persistent use of /etc/ld.so.conf.

Conclusion and Further Info

Extensive information about the actions used by the loader to use dynamic libraries are found in man pages: ld-linux.so, ldconfig and dlopen & friends.

John R. Levine’s “Linkers & Loaders” is an extensive depiction of linkers, loaders, libraries and the load process.

Proper knowledge of library management on a Linux based system relies on good understanding of the linking and loading processes and library types. Make sure you understand the advantages and disadvantages of each approach and choose the one most suitable to your specific needs.

This article is a quick guide to setting up a Python work environment. It walks you through installing Python with some basic package management tools (distribute, pip, virtualenv), setting up projects, and installing packages.

Bootstrapping

First of all we need to have a working Python interpreter. You want to install the latest release of 2.7 for now (September 2011). Python 3 is gathering momentum but many libraries don’t support it yet.

• In most Linux distributions, and in Mac OS, some Python is already installed. You may, of course, install a different one from scratch. For Mac OS, the homebrew version is highly recommended.

• On Windows you install a pre-compiled release from http://python.org/download/.

• To install from source, you need a C compiler, and a tarball from http://python.org/download/. The usual ./configure; make; make install should work just fine. Consider installing into a separate folder, e.g. ./configure --prefix=/usr/local/Python-2.7, so you can easily remove it at some point in the future.

Now, the typical mistake is to declare victory, and use this Python installation for everything. In time, you want to use various libraries, so you install them on top of Python. Eventually you get a version conflict (some project requires a library which is too new for another project). Fortunately there is a better way: virtualenv.

The command-line examples use $MYPYTHON as placeholder for the Python installation path. This can be /usr for a Linux distribution install, /usr/local for default manual installation, /usr/local/Cellar/python/2.7.2 for mac Homebrew, or even C:\Python27 on Windows.

If you’re on Linux, and use a Python package from the distribution, it’s a good bet they have virtualenv too. For Debian, Ubuntu and Fedora, the name is python-virtualenv. This may be outdated, so if you experience problems, check the version and consider installing the latest one (see below).

In a fresh Python installation, to get virtualenv, we need to install distribute and pip first. distribute is an older package manager, and pip is newer and more powerful, but it depends on the older one to do heavy lifting. So, download distribute_setup.py, and, assuming you installed Python in a folder called $MYPYTHON, do the following:

> $MYPYTHON/bin/python distribute_setup.py
> $MYPYTHON/bin/easy_install pip
> $MYPYTHON/bin/pip install virtualenv

If everything worked out fine, you should have a script called virtualenv in $MYPYTHON/bin, and you can safely remove distribute_setup.py and distribute-x.y.z.tar.gz.

That’s all you normally install in the global Python folder. Maybe throw in some commonly-used, slow-to-change, takes-a-while-to-compile package like PIL or SciPy, or the odd manually-installed kits on Windows, but everything else goes into a virtualenv.

Virtual insanity

Say you want to work on WoUSO, and the documentation tells you that you need to install Django. The very first thing you do is create a virtualenv. We’ll use $MYENV as placeholder for the path to a new folder where you want to work:

> $MYPYTHON/bin/virtualenv $MYENV

virtualenv will create the folder, write some files, then run off and get distribute and pip, it should all take a few seconds. When it’s done, you have $MYENV/bin/python, which is a fully functional Python interpreter. Next to it, there is $MYENV/bin/pip, which you can now use to install things:

> $MYENV/bin/pip install Django

This will go to PyPI, look for a package named Django, and install the latest version. The installation happens inside $MYENV, in the lib/python2.7/site-packages subfolder. This Django doesn’t affect the original Python installation or any other virtualenvs you create. Of course, multiple virtualenvs can have different versions of Django.

Bits and pieces

Now, if you start happily creating many virtualenvs, installing a lot of packages, you’ll be downloading the same files over and over again. Fortunately, pip can be configured to cache the downloads:

> cat ~/.pip/pip.conf
[global]
download_cache = ~/.pip/cache

Depending on the setup, sometimes you have to deal with globally-installed packages, for example if you’re using the Python from a Linux distribution. It’s still possible to create a virtualenv that ignores those packages by passing the --no-site-packages option to virtualenv. This simply leaves out the global site-packages folder from Python’s import path.

Some projects include a requirements.txt file in their source tree, which lists dependencies. You install these with pip install -r requirements.txt. Writing your own requirements.txt is easy: each line is a set of arguments for one invocation of pip. Or simply run pip freeze, it generates a list of all the installed packages and their versions.

When you get tired of typing $MYENV/bin/something all the time, you may want to activate the virtualenv. This is a fancy name which simply means that $MYENV/bin is prepended to your current $PATH (and your $PS1 is enhanced):

> . $MYENV/bin/activate
(myenv)> # "python" invokes "$MYENV/bin/python"
(myenv)> deactivate
> # back to the original shell environment

If you find yourself working on a package, the kind that has setup.py and installs with pip, you want to install the package in “edit” mode. Check out the source tree, then (assuming you’re in the same folder with setup.py) run pip install -e .. This will install the package in-place. Technically, a link is made in site-packages that extends Python’s import path to find your package, any dependencies in setup.py are installed, and scripts are installed in $MYENV/bin, if the package has any.

Further reading

These wonderful tools are available on PyPI, the Python Package Index. Most of them have good documentation that explains more features that did not fit in this article. Also, remember docs.python.org (behold the table of contents), where you can find documentation on the language, a nice tutorial, and excellent documentation for the standard library.

Git is an excellent SCM (source code management system). I use it for a plethora of tasks such as managing code, scripts, LaTeX files, config files, Org-Mode files. I try to base all my actions on text files such that it could be managed through Git.

In this post I wish to share some of the knowledge and skills I’ve gathered throughout the time of using Git. I am novice myself in many aspects of using Git, but I feel confident of my basic usage skill and good practices.

My aim is to present tips and good practices that allow using Git at its value and conforming to recommendations. This is not a tutorial or a comprehensive view of Git. In case you are looking for that I recommend the excellent Gitimmersion tutorial and the Pro Git Book.

An important aspect to have in mind is the data model that Git uses. While most SCMs use changesets to manage commits, Git uses snapshots. Each commit is a snapshot of the entire project; it is not a set of file patches. Bear this in mind when using Git commands and playing around with commits. You may also check this tutorial for a more thorough presentation.

Configuring Git

The first step of using Git is configuring your identity and preferences, as highlighted by most tutorials. The recommended practice is to configure Git at system level (using the --global option):

git config --global user.name "Razvan Deaconescu"
git config --global user.email "razvan.deaconescu@cs.pub.ro"
git config --global color.ui auto

I recommend issuing the above commands each time you are using an account that will make use of Git commands.

In case you want a different configuration (another email address, for example) for a given repository, just issue the above commands (sans the --global option) while in that repository.

A situation may arise when you want to create a commit (or a series of commits) that use different user information. This may happen when you and a friend have access to a common account, and you want to separate your commits form hers/his (although run from the same account). There are two situations and approaches to this:

1. Situation: You want to use a different identity for all (or most) commits in a shell session (such as an SSH login session). Solution: Define the GIT_AUTHOR_NAME and GIT_AUTHOR_EMAIL environment variables:

   export GIT_AUTHOR_NAME="Mighty McWolf"

   export GIT_AUTHOR_EMAIL="mighty@mcwolf.org"

2. Situation: You want to use a different identity for a single commit. Solution: Use the --author option when committing:

   git commit --author "Mighty McWolf <mighty@mcwolf.org>"

Commits

Everything in Git revolves around commits. A commit is a basic unit of information that you submit to Git for handling. Git stores each commit and links it to other commits such that you see a commit history, get back to a previous state, create a branch, watch the commit tree, update certain commits, create tags and many others. As mentioned above, a commit represents a snapshot of the entire project.

A basic rule, that applies to all other SCMs, is that each commit must keep the repository in a compilable state. That is, if one would checkout to a random place in the commit history, he/she would still be able to compile the source code. Make sure the project is in a compilable state when issuing your commit.

While the repository needs to be in a compilable state, it need not run perfectly. In fact it may end up in “Segmentation fault” or other critical errors. That’s no problem; it’s not achievable (not possible actually) to have a clean repository where each commit would break nothing. Do not be afraid to break the application when issuing a commit as long as its in a compilable state. If the application breaks, another commit will fix it; an impatient contributor could very well revert to a previous commit and create a branch from there. Moreover, trying to keep the application running, may force you to disobey the next recommendation.

Another important recommendation, heavily stressed in Git but probably insisted on in other SCMs, is creating small, atomic commits. Each commit should do one thing and do it well. A commit should not use a message such as “Update everything.” or “Fix plenty of errors.” Rather, each fix should go into a separate commit. This would make it very easy for a reviewer to analyze and diff your commit and, possibly, isolate a bug that you may have introduced. If your commit ranges a whole bunch of features that introduce multiple bugs, isolating those bugs and fixing them is a pain.

So, remember: Create small atomic commits that keep the repository in a compilable state.

Commit Messages

When your commit is ready, you’ll issue the git commit command and either use the configured editor or the -m option to write the commit message. Either way there’s a basic set of recommendations you should follow when writing a commit message.

1. Keep it short. Ideally, your commit message should consist of at most 50 characters. In case your message is longer, break it into sentences, and leave a blank line between the 50 characters message and the rest. The rationale, as mentioned in the git commit manpage, is that the first line is used as an email subject line by various tools.

2. Use present tense when issuing a commit. This ensures “compatibility” with messages used by tools such as git merge.

3. Write sentences not descriptions, similar to good code comments. Use capital letter, use verbs and end with dot.

Tim Pope writes about what makes a model Git commit message.

Creating and Updating Commits

Remember that your commits should be small: do one thing, do one thing well.

What happens when you’ve made a lot of changes and you want to create a commit? You need to “split” your changes in multiple commits. For that you use git add -i (-i for interactive). When using -i Git inquires you about the commit. Most likely you would:

1. choose the patch option (press p or 5)
2. choose the file you want to “split”
3. press Enter
4. answer y or n to include/exclude certain chunks
5. press q to quit

At this point, the modified file would be found both in the staging area and in the “changes” area. The staging area would solely consist of the chunks you selected previously.

What if you’ve just created a commit and realized that the commit message may be wrong or that there should have been another hunk or file committed? In this case you would use git commit --amend. As the options says, this gives you the possibility of amending the commit, be it to update the commit message or to add certain files: just issue git add (or git add -i) and then invoke git commit --amend. By adding --author, the --amend option allows you to even update the author identity.

What if you want to update a commit that is not the latest? If the commit has been pushed in the remote repository, then it’s quite complicated and not recommended. However, if the commit is local and hasn’t been pushed, you may used git rebase -i. You have to specify the commit id where rebasing will take place. Afterwards you will be prompted with an editor screen where you can select which of the commits that have been created. Usually you would replace the pick string with edit and Git will pass you through all commits.

For each commit you will most likely issue some git add commands, then git commit --amend and, finally, git rebase --continue.

As long as the commits are local (not pushed to the remote repository), all is fine.

Stashing

On certain occasions, you may need to run some commit update commands (such as git rebase, git pull) but retain some “dirty data” in the repository. As Git disallows the existence of non-committed data in such occasions, the solution is stashing.

Stashing means you temporarily store your data in a specialized zone such that it would not get in the way of the above commands. In order to stash local changes, you would simply issue the git stash command. After updates have occurred, use git stash pop to bring back changes and revert to the original “dirty state”.

Ignoring Data

Some files or data have to be ignored from being commit, while others need to be ignored because of process specifics or use preference.

As a rule of thumb, a repository should only manage text files; no binary files such as image files, compressed files, object files, executable files. If you are a web developer or someone who has to work extensively with image files, the above rule wouldn’t apply 100%. You should however, only commit source code files and files that cannot be compiled or linked from other files.

Such that a good practice is to create a top-level .gitignore file in your repository and define files to be ignored. A basic .gitignore file is shown below:

sample .gitignore

*~
*.swp
*.swo
*.o
*.obj
*.a
*.so
*.dll
*.lib
*.gz
*.bz2
*.zip

Optional .gitignore files may be created in subfolders of the repository according to need.

.gitignore files are committed in the repository and their exclusion rules are applied to all contributors. A situation may arise when you create a folder that you want to reside in your repository clone but never get committed. For example a lib folder consisting of libraries you are linking against for testing purposes. As it is binary data it shouldn’t be committed, and, as you are the only one using it, it should be ignored. You could add it to the .gitignore file but that would complicate it. The best solution is to edit the .git/info/exclude file. It follows the same syntax as .gitignore files but is local to your clone.

The above solutions are not useful in a specific situation: you want to ignore changes you make to a file that is being tracked. .gitignore and .git/info/exclude only ignore non-tracked files; they can’t be used on files that are being tracked. Your solution lies in running the command git update-index --assume-unchanged abc.txt. Issuing this command ensures that any local updates to the abc.txt file are not going to be taken into account when creating subsequent commits.

Viewing Git Information

A large part of your interaction with Git is analyzing commits, diffing, checking commit history etc. Visual tools are very important and provide you an intuitive view of the repository commits. Such tools are Git GUI, gitk and giggle. A nice tool, running on an ncurses-based interface is tig.

Apart from that, several commands are heavily used throughout your work in Git, from a “view point of view” so to say:

• git status provides you with information regarding the current branch, information in staging area, “dirty” information etc.;

• git log provides you with a CLI view of the commit history; an useful option is --oneline providing you with a one commit on one line view;

• git diff presents a diff between various states of the repository;

• without any option, git diff it shows changes in the working directory (versus HEAD);

• a single option to git diff is a commit ID or tag that is diffed against HEAD;

• two options tor git diff are two commit IDs or tags to be diffed.

An useful option to git diff is --cached. This option presents a diff between HEAD and data in staging area. It’s useful to check everything is in order before creating a commit.

Cleaning Up

An important activity is cleaning up files in different states (staging, modified, non-tracked).

The list below highlights various user requirements and solutions to those predicaments:

• You want to clear any updates you’ve done to a file that’s being tracked:

  git checkout file.name

• You want to remove a file from the staging area and place it in the modified state; you want to build your commit in a different manner:

  git reset HEAD file.name

• You want to clear non-tracked files from the working clone:

  git clean file.name

• You want to clear all non-tracked files from the working clone:

  git clean -f

• You want to clear all changes and revert to the initial state of HEAD (by changes I’m referring to tracked files changes; this doesn’t affect non-tracked files):

  git reset --hard

Other Resources

The Internet is filled with tutorials and tips regarding the use of Git. Google is one of your best friends to provide you a rapid solution to a problem. Through Google, I’ve found a lot of answers on Stack Overflow.

As mentioned above, I find the Git Immersion tutorial to be very well presented and easy to follow and the Pro Git Book as a good technical presentation of Git and its features. An excellent site, consisting of a plethora of very nicely presented tips is > learn git one commit at a time">> learn git one commit at a time">git ready.

As a funny link, I recommend you access Commit Message Generator.

This post will shed some light on the differences between arrays and pointers specifically when it comes about referencing string literals. We will base our discussion on the following two programs:

array.c

char a[] = "ROSEdu";
int main(void)
{
	a[0] = 'r';
	printf("%s\n", a);
	return 0;
}

pointer.c

char *p = "ROSEdu";
int main(void)
{
	*p = 'r';
	printf("%s\n", p);
	return 0;
}

Program array.c defines an array of char whose elements are initialized with character string literals, while pointer.c defines a pointer to char and initializes it with the address of a memory area holding a string literal. Notice array a and pointer p allocations in the image above. Can you make a guess about size of a and size of p? Next, both programs modify the first character of the string literal ROSEdu. Are these two programs equivalent? At the first glance the answer seems to be positive, but let’s have a minute and actually run the code.

$ ./array
rOSEdu

$ ./pointer
Segmentation fault

While we could modify array a, our program was killed attempting to modify string literal pointed by p. We will now have a look at the generated assembly code and notice the section where string literal ROSEdu is stored.

$ gcc -S array.c -o array.s
$ cat array.s
.globl a
        .data
        .type  a, @object
        .size  a, 7
a:
       .string "ROSEdu"

$ gcc -S pointer.c -o pointer.s
$ cat pointer.s
globl p
       .section        .rodata
.LC0:
       .string "ROSEdu"
       .data
       .type  p, @object
       .size  p, 4
p:
       .long  .LC0
       .text

We can see that array a is stored in data section, which is writable and there is no problem when it is modified. On the other hand, we can notice that p is a pointer stored in data section but it points to a read only memory location, thus accessing it results in ‘Segmentation Fault’.

C99 standards (Section 6.7.8) states that:

• contents of the array a is modifiable.
• if an attempt is made to use pointer p to modify the contents of the array, the behaviour is undefined.

So now we see why pointer.c program crashed. gcc decided to store string literal pointed by p into read only data section. One must remark that this is not mandatory, and its implementation dependant.

We invite you to answer following questions:

• What is the sizeof(p) and sizeof(a) in our previous examples?
• What happens if variables a and p are declared on the stack?
• Is it possible for the following expression (const char []){"ROSEdu"} == "ROSEdu" to yield true?

This blog started as an idea of Răzvan Deaconescu based on the fact that several members of ROSEdu already had technical blogs (linked here on the right) but they were not updated on a regular basis. Having a community blog solves this problem and allows for a greater diversity among the topics presented. Without further ado, this is it.

Excluding this article, the blog will contain technical articles, tips and tricks, quick hacks to solve some problems and some articles explaining different things related to IT. Here we will present the infrastructure of the site.

It is created using the Jekyll static site generator. When this was started there was a little discussion about the back-end and why we should choose a static site generator instead of using Wordpress or Blogspot or similar frameworks. The problem with the “classic” CMSs was that they have a (relatively) high demand of resources and that you’ll always have to use the latest version of the software to prevent attacks.

Using a static site there’s no database layer, no code which may contain exploitable bugs. Thus, everything is safe. Also, since only basic HTML pages need to be served, there’s little memory and CPU usage when serving the website.

Since we will be posting code to this blog, a framework allowing a nice and pleasant look of it was desired. And Wordpress has several plugins but it isn’t always fun working with them.

But, the really important aspect, the one which ended the debate, was the fact that we are able to separate the content from the actual site, we can keep a back-up copy of the content¹. Moreover, we can use a Git repository to hold the configuration and the content. We use a gitolite repository for the actual content with two branches:

• one for the actual content
• one for contributors where the actual content gets posted for review

After the review is done, the actual publishing is done by cherry-picking the good commits to the first branch. A git hook is responsible for doing the actual posting.

The content is written in a simple text file either using the Markdown format or the Textile format. Jekyll is responsible for translating it into the actual HTML page and some server configuration takes care of actual publishing of the pages.

There’s one downside to this, though. Comments and other things that need dynamic code are hard to set up. Yet, there are solutions there too. For example, we use the Intense Debate platform for comments. The alternative, Disqus could also be used but we settled on the first.

The actual design of the site was done by learning and experimenting with CSS until we were satisfied with the looks. Having a static generator means that we had to do the entire design by hand but this can be a good thing since we no longer depend on the themes that come with a classic CMS.

Thanks go to: Alex Juncu (setting up the Apache stuff), Răzvan Deaconescu (coming up with this idea and setting up the initial repository), Mihai Maruseac (configuration, layout and workflow)

If you wish to contribute, contact us at techblog@rosedu.org.

This is another test. Nothing to see here.

Done while setting up repo. Hopefully, it works as indented

This week was a good week for coding, and the best part about it: most of my changes are now in s-c’s upstream trunk.

I started out trying to populate s-c database with only a few applications; this meant updating the update-software-center tool, and also changing some bits on the database update module. I was glad to discover that some functionality I have planned to develop in week 6 (parsing AppStream app-data xmls) was already there, thus making my job easier

After that, work continued by abstracting backend parts, such as InstallBackend and TransactionsWatcher, and also isolating as much as posible apt and apt_pkg usage. I have then expanded the PackageInfo abstraction to implement a dictionary like interface, containing _Package objects (before that, apt.Package were returned). It took some small steps, and sometimes mistakes from my side, but in the end, I guess I have got it done right. Work still needs to be done regarding usage of candidate and installed properties (which currently are apt.Version objects), and also on AptCache try_install_and_*depends methods (this should be either abstracted, either made apt-backend specific.

At the end of the week, preparing the next point of focus, I have managed to get a developing environment with PackageKit with Python gobject introspection data. Here comes the bad news: the py GI for packagekit is not ready for prime time. Reasons for that: GI is relatively new, there are no other users of PK from Python (only C and Vala). With help from ximion, hughsie, dantti and tomeu (and many other kind souls on #PackageKit, #introspection or #python), we isolated the problem inside pygobject, apparently an incomplete implementation of GPtrArray. I hope to get it fixed next week , so that I can continue work with PK.

Although I am a bit worried seeing segfaults in Python, it’s a sunny day out here (so I can finish my report on a non-technical note) so I’m sure that with help from these awesome people on IRC,  problem will be solved, and my project will continue according to the plan.

Next week: gi ninja and more PK!

Not much to say about this first week of gsoc (Exams period hasn’t finished yet), so here it comes:

- had an IRC meeting with mentor vuntz of openSUSE and mvo of Ubuntu (the initial software-center author), got some things cleared up and planned the development

- according to my timeline, I now have the trunk version of software-center running in Debian wheezy; there is still work to do, but is an encouraging start;

next:

- work on a dummy install backend and a dummy package info provider; probably create an elegant way of switching backends;

other stuff:

- got interviewed for the openSUSE news

This is it. See you next week.

Hi everyone,

Just a quick announce that I will be working this summer as a GSoC student, for the openSUSE Project. My mentor will be Vincent Untz, the so called “father of GNOME 3″

What I’m aiming to do is modify Ubuntu Software Center to use PackageKit (an universal package management toolkit) as a installation backend, and also integrate it with the AppStream initiative (a cross-distro project for making software installation easier).

My full proposal can be read here (fixed). Feedback and suggestions are, as always, appreciated.

I would like to thank ROSEdu (which I am a member of) for the support

I have added a RSS feed to this site. This article is used to test it.

Also, this is used to test textile markdown.

Lists:

• this is an item in a bulleted list
• this is another item, with an ordered sublist
  1. 1
  2. 2
• this is the end of the list, with a footnote¹

Tables:

Code:

Simple:

a = a + 1;

And complex

def info(self):
	print "This is a piece of code"

¹ In textile, the footnote doesn’t have a back link. Also, it is written
differently. See http://mojomojo.org/documentation/textile_vs_markdown for a
good comparison between the two.

I have added a RSS feed to this site. This article is used to test it.

Also, this is used to test textile markdown.

Lists:

• this is an item in a bulleted list
• this is another item, with an ordered sublist
  1. 1
  2. 2
• this is the end of the list, with a footnote¹

Tables:

Code:

Simple:

a = a + 1;

And complex

def info(self):
	print "This is a piece of code"

This is another test. Nothing to see here.

This is a test post. See if you really like it before deciding if we should keep this format. Same content as the above post. Used to test several layouts.

Why did I chose to use a static site generator instead of using Wordpress? Becuase wordpress has some thing which bother me. For example, in Wordpress it is very hard to include source code in a proper way. However, using Jekyll, we won’t be limited by Wordpress¹. Really, our only limit is the sky².

A post is simply a file with Markdown syntax. If you don’t like this syntax, you can try Textile syntax. Or, someone could post your plain text entry.

Now, some code samples to see how this is working. First, something in Python.

def info(self):
	print "This is a piece of code"

And, with highlight:

def info(self):
	print "This is a piece of code"

Now, something in Haskell, too (try to guess the project :P)

convert file = do
  s <- readFile file
  let sections = map parse . split $ s
  let ast = combine sections
  let out = output ast
  putStrLn out

Finally, some C code

#include <stdlib.h>

int main()
{
	return 0;
}

And now, we’ll end this post. However, there are a few foot-notes, aren’t they?

This is a test post. See if you really like it before deciding if we should keep this format.

Why did I chose to use a static site generator instead of using Wordpress? Becuase wordpress has some thing which bother me. For example, in Wordpress it is very hard to include source code in a proper way. However, using Jekyll, we won’t be limited by Wordpress¹. Really, our only limit is the sky².

A post is simply a file with Markdown syntax. If you don’t like this syntax, you can try Textile syntax. Or, someone could post your plain text entry.

Now, some code samples to see how this is working. First, something in Python.

def info(self):
	print "This is a piece of code"

And, with highlight:

def info(self):
	print "This is a piece of code"

Now, something in Haskell, too (try to guess the project :P)

convert file = do
  s <- readFile file
  let sections = map parse . split $ s
  let ast = combine sections
  let out = output ast
  putStrLn out

Finally, some C code

#include <stdlib.h>

int main()
{
	return 0;
}

And now, we’ll end this post. However, there are a few foot-notes, aren’t they?

La începutul lui septembrie mă aventurasem într-un proiect în Haskell așa cum era Hammerfall-ul (ultimul post de aici e prea optimist ca să-l linkez în titlul proiectului, atunci nu părea că ar fi picat) din ROSEdu pe vremea când eram în anii 1 și 2.

Cu toate că aveam destul de mult entuziasm (prima versiune a fost terminată în 2 ore de la începerea proiectului) timpul a făcut ca HaCoTeB să eșueze. Ceea ce înseamnă că acest proiect de pe github nu va mai fi continuat.

Postmortem pot spune doar că e nasol să începi un proiect cu un set de idei, să te răzgândești pe parcurs și între timp să ți se dea prea mult de lucru din alte părți astfel încât proiectul să-ți moară din cauză că ai uitat ideile sau că ai găsit o versiune mai interesantă.

Java killed HaCoTeB dar am o idee totuși. O voi pune în practică când voi posta următoarea bucată de cod, probabil când voi începe următorul proiect.

Sâmbăta aceasta, ROSEdu organizează a doua prezentare din acest an din cadrul ROSEdu Tech Talks. Este o prezentare de LaTeX care, cred, va interesa destul de multe persoane care vor dori să redacteze documente în mod profesional (de exemplu un articol de licență sau un articol pentru o publicație). Vă așteptăm.

Dacă vă întrebați cu ce am fost busy câțiva dintre noi zilele astea, faceți click pe

Content is still WIP.

Ne vedem diseară. Work to do

Ieri, ziua 42-1 (treceți în binar și puneți puncte din 2 în 2), ROSEdu a avut 2 evenimente în facultate: Linux Install Fest (ediția a patra din Ubuntu Install Fest) și ROSEdu Tech Talks, prima prezentare din sesiunea a doua de prezentări.

Deși era planificat să înceapă de la ora 10, doritori pentru LIF au ajuns în facultate de pe la 9.30. Până pe la 10-10:15 EG306 era overcrowded așa că am aplicat o strategie de lucru distribuit și am folosit și EG405. În acest timp, se ardeau CD-uri și se făceau stick-uri USB bootabile pe bandă și se instala Linux la doritori.

Cred că au fost și câteva persoane cărora li s-a instalat și altceva în afară de Ubuntu 10.04: câteva persoane au vrut sau au fost nevoite din cauza unui bug să pună 10.10 (care apare azi:P), la un moment dat s-a încercat și o instalare de Debian, câteva persoane au instalat OpenSuse de pe CD-urile originale aduse de reprezentanții comunității OpenSuse.

Pe la 12, toată lumea s-a mutat în EC004 pentru a asista la prezentarea lui Răzvan: «Utilizarea eficientă a CLI-ului». În sală erau maxim 10-20 locuri libere, deci se poate zice că am avut cea mai mare prezență și la un Tech Talks, nu numai la un LIF. Prezentarea poate fi găsită pe site-ul dedicat evenimentului, după ce AJ o va încărca acolo :)

După ora 1, ne-am întors în EG-uri să finalizăm instalarea la câteva sisteme. Tot procesul s-a terminat undeva pe la ora 6 după ce ne-am prins urechile în câteva incompatibilități pe care am încercat să le rezolvăm.

Lista tuturor participanților poate fi văzută pe noul site ROSEdu, aici.

Vă așteptăm și la evenimentele următoare, inclusiv la posibilele surprize care se pregătesc :D

Începe un nou an universitar și multe lucruri se înoiesc și se adaptează la noile condiții și noile tehnologii. ROSEdu a lansat o variantă nouă a site-ului, una care va fi ținută mai des up-to-date și care va avea mai mult conținut relevant. În curând, se va adăuga și mai mult conținut decât este acum, ajungându-se până la publicarea a 100% din ceea ce facem.

Dacă doriți să participați în activitățile noastre, nu ezitați să vizitați pagina de contact.

PS: Sâmbătă se vor porni multe dintre proiectele ROSEdu: Linux Install Fest, ROSEdu TechTalks și un Open Source Lan Party. Sunteți invitați în EG306 pentru cele de mai sus precum și pentru prezentări din partea reprezentanților câtorva asociații Open Source din România (distro related sau generale).

PPS: Dacă doriți să participați la generarea de conținut pentru WoUSO (și ați participat la o ediție anterioară a jocului), vă rog să trimiteți un mail la mihai dot maruseac coadă-de-maimuță gmail dot com.

O veste bună din facultate, câteva teasere și anunțuri.

Anumiți profesori (excluzându-i pe cei la care țin ore, mă refer aici la cei pe care nu-i vedeam în stare de așa ceva) chiar se uită pe feedback și interacționează corespunzător. GG :)

Depus cerere de schimbat curs nașpa cu un curs super. Vedem în cât timp se rezolvă.

S-a dat un anunț pentru colocviul de practică. Dacă nu știți când e, contactați șeful de grupă, ar fi trebuit să primească un mail cu planificarea.

Sâmbătă vă așteptăm la Linux Install Fest (cunoscut și ca Ubuntu Install Fest, acum ediția a 4-a). ROSEdu TechTalks pornește tot sâmbătă.

Se pregătește o surpriză pe site-ul ROSEdu. Și încă una aici (cam într-o lună două).

Aveam o meditație după cursul de azi de AI dar o păstrez pe săptămâna viitoare până mai strâng idei ca să fie ceva mai lungă.

Am avut un concediu de 2 săptămâni. Am fost la Călimănești, la a patra ediție a IP Workshop (IP = Ideas and Projects). Cursul de programare pe Android și Blackberry ținut de Alex Radovici.

Overall, a fost super tare și mi-au plăcut atât cursurile cât și activitățile extra (mici excepții listate în feedback).

PS: Mi-am cam pierdut talentul de a scrie chestii de lungime normală. Astea ies mult prea scurte (a 5-a încercare de a scrie ceva)

Buna ziua,

Va invitam la a IV-a editie a Scolii de vara IP Workshop. Anul acesta,
ne-am mutat la munte, pe valea Oltului. Asadar, va asteptam in perioada *30
iulie – 12 august* la *Grupul Scolar Economic* din statiunea *Calimanesti*,
judetul Valcea. Echipa noastra s-a deplasat la fata locului asa ca pe
pagina www.ipworkshop.ro sau calimanesti.ipworkshop.ro puteti gasi
fotografii cu conditiile de cazare.

Lista cursurilor porpuse anul acesta este:

- Fotografie
- Grafica si Design
- Retele de calculatoare
- Programare pentru Android si Blackberry
- Robotica
- Utilizare Linux
- Dezvoltarea proiectelor software

Pentru participarea la scoala de vara, va trebui sa va inscrieti la *un
singur curs*. Totusi pentru a nu va obliga sa va limitati la un singur curs,
anul acesta am creat ideea de seminarii. Acestea sunt cursuri de una sau
doua zile, cursuri ce nu se vor tine in paralel si la care poate participa
oricine. Incrierea se va face la fata locului. Pana acum, seminariile
porpuse sunt:

- Adobe Flash
- Probleme de securitate
- Python si Django

Lista nu este definitiva, asteptam si propunerile dumneavoastra.

Cazarea si masa vor fi asigurate de catre gazda noastra, Grupul Scolar
Economic din Calimanesti. Conditiile de cazare sunt bune, camere de doua sau
patru paturi cu baie la camera. Fotografiile se gasesc pe pagina de
Internet.

Costurile participarii incep de la *500 de lei* pentru elevi si studenti
si *300 de lei* pentru profesorii insotitori. Acestea includ cazarea, masa
si excursiile. *Acestea sunt valabile doar daca plata este efectuata pana la
data de 1 iulie. Ultima zi de inscriere si plata a taxei este 20 iulie.*

Pentru mai multe informatii, va asteptam pe pagina noastra de Internet,
calimanesti.ipworkshop.ro, www.ipworkshop.ro, pagina de Facebook sau prin
email la info@ipworkshop.ro.

Avem de asemenea o rugaminte catre dvs., si anume sa popularizati cat mai
mult acest mesaj.

Va asteptam cu drag,
Echipa de organizare
—

Salut!

Vine vara și te gândeşti să faci practică? Ţi-ai dori să lucrezi la un proiect open-source?

Dacă răspunsul este afirmativ, ROSEdu[1] îţi oferă ROSEdu Summer of Code[2], aflat acum la a treia ediţie. RSoC este un program de lucru în timpul perioadei de practică la un proiect ROSEdu. Proiectele în care vă puteți implica acoperă o gamă variată de necesități, limbaje de programare, interfețe de utilizare, etc. Veți avea ocazia să fiți membru al unei mici comunități de dezvoltare open-source folosind produse și utilitare tipice unui astfel de proiect: sistem de versionare, compilatoare, interpretoare, wiki, bug reporting, liste de discuții, IRC, site, etc. Dacă veți fi foarte entuziaști și doritori, puteți continua colaborarea la respectivul proiect și după finalizarea perioadei de practică.

Pentru alte întrebări sau detalii consultaţi rubrica FAQ[3]. O listă și o descriere a proiectelor poate fi găsită aici[4].

Pentru înscriere folosiţi pagina de înregistrare[5] sau pagina ROSEdu de pe Stagii pe Bune[6]. Perioada de înscriere continuă până pe 10 iunie.

Vă aşteptăm,
Echipa ROSEdu

[1] http://rosedu.org/
[2] http://soc.rosedu.org/2010/
[3] http://soc.rosedu.org/2010/faq
[4] http://soc.rosedu.org/2010/proiecte
[5] http://soc.rosedu.org/2010/inregistrare
[6] http://www.stagiipebune.ro/detalii-companie.html&id=172&category=49

A fost și seminarul Ixia (al 4-lea dacă mai țin minte de la început, dar primul la care am putut participa – orar stupid în ceilalți doi ani). Live blogging (twitter și Facebook) în timpul lui pentru a avea notițe pentru acest entry în care-i voi face un rezumat.

Prima prezentare i-a aparținut lui Andrei Pitiș și a atins diversele opțiuni pentru o carieră în software engineering. Unele libertine dar fără garanția unei pensii (după cum a remarcat Alexandra:P), altele riscante dar cu mult fun și multe beneficii iar altele ok dar în care avansarea se face foarte foarte lent (depinzând de ceva SIGKILL-uri :) ).

A urmat Alex Badea care a prezentat viața unui software engineer la Ixia și apoi Andrei Lascău pentru partea de Q&A. Astfel, am aflat că la Ixia cafeaua de orice sortiment este nelimitată și că au echipamente ce pot genera trafic echivalent cu descărcarea a peste 15 DVD-uri HDD în câteva secunde (numerele mai mari sunt greu de ținut minte).

Au fost studenții de la diplomă ce și-au prezentat proiectele realizate în colaborare cu Ixia: Vali Priescu cu o tehnică de freeze pentru hard disk, Vladimir Olteanu (aka koliva) cu o altă tehnică ce a amintit puțin de Haskell și structurile lui Okasaki, Sorin Dumitru și Cătălina Macaleț cu un simulator de rețea peste Linux Kernel Library și Andrei Buhaiu, Vlad Dogaru și Lucian Cojocar cu un simulator al interacțiunilor dintre procesele unui sistem de operare.

La final, s-a organizat o tombolă dar nu știu ce au primit cei 5 câștigători. Și s-au premiat câștigătorii Ixia Grand Challenge (cei care au realizat tema 5 la SO2 până azi dimineață pe ambele platforme într-un stil foarte elegant și fără nici o eroare detectabilă din vmchecker-ul lui Lucian Grijincu și Alex Moșoi (plus echipa de dezvoltare)): 2 HTC Hero pentru 2 persoane din cele 7 care au reușit să se încadreze în concurs – au terminat tema complet.

Termin aici pentru că vreau să folosesc fereastra asta de o oră și-n alte scopuri.

Articol postat din sediul ROSEdu, aflat în laboratorul Ixia din Politehnică :P

Aveți un proiect neterminat? Aveți o idee de proiect dar nu aveți echipă?

În acest an, ROSEdu vă oferă șansa de a vă propune ideea de proiect pentru ROSEdu Summer of Code. Se va lucra la proiect pe durata oficială de practică din facultate într-o echipă ce va conține cel puțin propunătorul ideii și un mentor din partea ROSEdu.

Pentru a participa astfel, trebuie să propuneți proiectul aici[1] până pe 27 aprilie.

[1]: http://spreadsheets.google.com/viewform?formkey=dGRSTnJGYXhJRXYzQ1h6S1hyRnpub1E6MQ

Ce faceți în vacanță? Stați la plajă sau va pregătiți pentru carieră?

Detalii în curând

S-a mai terminat o etapă a concursului. Am avut o soluție trimisă (Valentin Dobrotă) și 2 sugerate dar netrimise. Prin urmare, etapa de azi va avea 2 probleme. Nu uitați că dacă trimiteți soluția la oricare din ele ea trebuie să fie într-o paradigmă non-imperativă. Deadline: 15 martie, ora 23.00

Prima problemă are o cerință extrem de simplă: Dându-se două numere și fie numărul numerelor naturale de cifre cu cuma lor . Afișați știind că numărul de cifre e limitat superior de 4000.

A doua problemă e următoarea. Avem un set de task-uri de producție. Pentru fiecare task știm un cost fix pe care-l plătim dacă-l efectuăm cel puțin odată. Mai știm și un cost care va fi plătit pentru fiecare instanță a taskului. Astfel, costul total pentru n execuții este de n ori costul pe instanță plus un cos fix.

Mai știm că un task are nevoie de o listă de obiecte ce pot fi obținute din efectuarea altor taskuri. Avem un fel de rețetă și știm că ne trebuie x obiecte obținute din tx, de exemplu. Mai avem pentru fiecare task și un profit ce s-ar obține dacă nu folosim rezultatul lui.

Ca lucrurile să fie frumoase, avem și o limită de timp.

Se cere să scrieți un program ce va oferi o planificare a taskurilor pe un singur fir de execuție astfel încât la final să avem profit maximal.

Spor.

Acum că a trecut și penultima sesiune din ultimul an de facultate, iată sunt cele două proiecte la care am lucrat în timpul liber; care timp deși nu a fost mult, a existat (C3 nu e așa de crimă pe cât mă așteptam, myth busted).

Primul este unul personal, are legătură și cu lucrarea de licență, deocamdată nu public cod, spun doar că se prevede o aplicație măricică Python construită peste o bază de date SQLite, folosind sqlobject, GTK+, goocanvas, reportlab și mulți alți clopoței și fluierași. Am ajuns să fac – printre altele – widgets (prietenii știu de ce spun asta), și sper ca pe măsură ce o dezvolt să mă familiarizez cu aceste tehnologii. E o provocare să o construiesc și să o dezvolt și totodată o distracție.

Cel de al doilea proiect căruia i-am acordat mai mult timp în ultima perioadă, este portarea World of USO din PHP în Python – Django. Am luat alături de Vlad și Sergiu această decizie, deoarece nu eram mulțumiți de starea codului actual al WoUSO (prea multe contribuții de la persoane diferite, stiluri și abordări diferite); ca să rezum, era greu de dus mai departe, îmbunătățit. De ce Django? Pentru că Django e fun, prea multă lume l-a vorbit de bine ca să nu-l încercăm. L-am încercat și a dat roade: în două săptămâni am reușit să implementez o versiune de bază care acoperă aproape trei sferturi din funcționalitatea jocului. Tocmai am primit mail de la Vlad(um), este dispus să aloce timp, cred că vom face treabă bună în continuare .

WoUSO este unul dintre proiectele propuse pentru dezvoltare echipelor la Cursul de Dezvoltare Liberă. Mai multe detalii vor fi publicate pe site-ul cursului. Pot doar să lansez doar un zvon că s-ar putea să lucrăm la un modul WoUSO pe acel site care începe cu feis și se termină cu buc.

Dacă citești aici și te interesează unul dintre django, python sau wouso, nu fi timid, dă un mail. Ne-ar plăcea să te alături echipei .

Cam atât. Programming must be fun. Sunt limbaje precum chitonul care fac programarea frumoasă, părerea mea.

Links:

http://dev.rosedu.org/wouso/wiki/DjangoPort

http://wouso-django.rosedu.org/ (cont alex:alex)

Asta mi-a strigat unul în ureche… Oana era ocupată: “…avem Mac, KFC, pizza nuștiucum, patinoar…” spunea ea la telefon unei alte prietene (să-mi fie permis: arfe), ca recenzie a noului mall.

E vina mea, m-am gândit… m-am dus ca vițelul să văd cum arată AFI Palace, chiar în ziua deschiderii. Ce-i drept, e mare și se populează rapid… cu magazine și băieți care-și pierd aifonul pe la fete prin buzunare.

Vroiam să scriu un articol despre intrarea în ritm… este un sport nou pe care îl practic de vreo 4 săptămâni încoace. Pot zice că merge bine, mai bine decât chitara. Anul 4 la C3 înseamnă compilatoare și spârc. Am mai ales în plus apepe de la NT, și SI cu Tănase. Suficient de interesante și timeconsuming. Mai e BD și MP…

Duse sunt seriile, CA, CB și CC… mândria de a fi la CC, parcă suntem alți oameni acum, grupați pe specializări (not). Un lucru de care îmi pare rău (Jancă ar fi trebuit să spună mai multe despre asta până acum ), este că am pierdut Cercul temei. Cercul temei, acea manifestare socială (printre puținele la CS), în care studenții se adună roată și vorbesc despre teme. Care cum a terminat-o mai repede, ce bube s-au strecurat în enunț, despre cum sunt stupizi cei care au publicat-o ș.a.m.d..

Revenind la ideea cu ritmul (am început articolul acum 2 ore, am uitat demult de el), pe  lângă toată școala asta multă, îmi place că săptămânile încep cu cele două seriale de care m-am deprins: Curb your Enthusiasm și The Big Bang Theory. Ah, a mai fost și studențiada… varii motive au lăsat-o să treacă pe lângă mine.

Ah, ca să nu uit, azi am fost la TechTalks pe care îl organizăm (ROSEdu) și mi-a plăcut. Git is the new emacs. GJ Mircea.

Pentru că abia a trecut ACM-ul, mâine este UIFv3, nu am mai scris de ceva timp cod, iar AJ mă bătea la cap cu întreba despre un nou xsplash de modificat (după cel de la USO), am scris un mic GUI peste imagemagick care micșorează numărul de pași pentru personalizarea ecranului de login.

Arată așa (ignorând Terminatorul din spate):

și face cam ce zice.

Exemplu pentru laboratorul de USO.

Sursa: xsplash-theme-editor-0.1.tar.gz .

LE: v0.2 cu surse și pachet .deb

Enjoy! Btw, the new ubuntu is awesome.

Pentru că poți, pentru că nu vrei să fi prost (îmi place gluma), pentru că merită

Notă: pentru că unii nu au înțeles acest lucru, această campanie NU este o campanie anti-microsoft, anti-windows7, anti-Fii-deștept, anti-comercial, anti-bunsimț. Dimpotrivă, este o campanie pro free software și atât No hard feelings.

World of USO ended this week, and all culminated today, with a well-organised ceremony for awarding the prizes. The attendance was poor (the organisers and the contest winners), but I couldn’t have felt more at home; the geek per square meter concentration was through the roof. After a brief bit of roleplaying and short speeches from the original proponents (both currently involved in teaching USO — Usage of Operating Systems), most of the dev-team went for a drink; celebration was short but fun. Of course, those of us who felt they had nothing better to do stayed on a bit longer.

A bit of extra info on World of USO: It is the first major success of Rosedu, a browser game which involved competing with others in answering questions about the USO class. It also featured a Quest mode, which was more about ingenious solutions and googling skills. Three special quests were also launched, the last of which was quite brain-twisting — only two people solved it.

Read the gritty details (in Romanian) on the blogs of Alex and Mihai or check the pictures at the album.

I got rejected for Google Summer of Code, but that was to be expected. As easy as Plan 9 is and as much as I loved working with it at the beginning, such matters ar too serious to be covered within a week. Perhaps next year I will be better prepared in a field; I would of course like to try my hand at it again.

But Razvan (my Operating System Usage teacher in the first semmester) came up with a proposal to write a system that generates spreadsheets for hourly paid course assistants in our faculty. I naturally agreed and a team was quickly formed. We met today for the first time. While the project itself doesn’t sound like much, we decided to do it properly. We are using Razvan’s server for the entire development process (I got a new IMAP email address courtesy of him), including mailing lists, RCS (to be decided), web page, wiki and testing.

Everyone is very excited and we’ve already outlined a brief design: a C library for reading a configuration file and actually outputting the spreadsheets, a (probably PHP, which I’m not happy about) web interface, a minimal, console based program, which will be called by the web interface, and classic, offline programs for both Windows and Linux. We are considering GTK or wxWidgets. The latter looks better, but I’m reluctant to use C++ and it has no C bindings. The spreadsheets will be XML-based, using ODF and Open Office XML, but we’re planning to keep the config file simple (no XML, as parsing it would probably be harder than generating the documents).

For the moment, we still need to find a name, although a mailing list is in place. A wiki and versioning system will follow. I’ve chosen to read up on creating dynamic libraried and to think of the configuration and console program format. Razvan also suggested using a lexer, which is an entirely new concept to me (actually it was until a couple of hours ago).

Amazing how much a little planning can save; we probably would have switched ideas a lot of times and still wouldn’t have come up with a design close to this one. Still, as good as it seems, it will probably be subject to change once coding gets off the ground.

We now have a name for the previously-unnamed-project — cspay. I prefer not to capitalize it, although some of my friends write it as CSpay or CSPay. The meeting today was fruitful and quite fun, with delightfully opposing opinions between the C guys (me, Razvan and Luci) and PHP folks (Alex and Mihai). Roxi and Andrei (andrew, whatever…) took a strategically quiet position, probably snickering at our endless quarrels. However, had we agreed on things from the start, it would surely have been a wrong design; hence the long discussion about the inner workings of our project were beneficial, or so I like to believe.

I got the task of designing a structure that represents a spreadsheet, and then writing the lowest of the libraries, converting that structure to an XML file. Not a particularly challenging task, algorithmically speaking, but interesting nevertheless. After the aforementioned hour-long discussion, we settled that the PHP scripts would also read the master cinfiguration file. Thus, we will write code that does the same thing twice: in PHP and in C. I’m not quite happy about that, but the alternative was getting really complicated; not complex, but complicated. And that I want to stay away from.

Once the name had been decided, Mihai wrote a simple and suprisingly functional IMO sketch, and Razvan quickly provided the space for it and a development wiki. Things are getting going, with most of the initial setup in place, save for a RCS. But we haven’t written any code yet (and will not do so at least for another week probably), so that is a non-issue.

We found an ini parser for C, there is a link for one in PHP on the wiki, so we are looking towards simple ini file for the configuration. We also found xmlindent, which is unmaintained (apparently died at 0.2.17 a couple of years ago), but functional and useful; we are thinking of adopting or perhaps forking the project as a side-task, maybe on the long run.

Things are exciting, and I hope they will stay that way — I am close to the usual “screw it all” phase of any project I start working on, but hopefully everyone will motivate each other.

Because cspay planning is advancing steadily (or so it seems to me), Razvan installed Subversion. It proved to be quite tricky though. It’s probably a security measure, but a “plain” Subversion repository (in that just a plain svn create was run) cannot handle multiple users. So we spent about two days experimenting with it, until Razvan finally set the right permissions for the folder (his time is even more limited than ours, obviously). I wrote a quick post-commit email notify bash script (wow, suprisingly, I could do that); all that remains right now is to actually start coding; I should write the base for libspreadconv, the bottom library which creates an OpenOffice XML spreadsheet from a generic structure (which should be decided upon). There is a lot of reading to be done, and quite little time, with late mid-terms and all.

Exciting, yes. Tiresome, you bet.

I actually wrote my first lines of code for cspay these days, but they were mere headers for the library I am writing. I started reading parts of the standard and making a rough sketch of what will have to be included in an ods file; the standard is huge, but hopefully things can be simplified to a bare minimum, stock proto-spreadsheet. I am having difficulties deciding what to take for granted and what to expect from the user. libspreadconv has to strike a balance between being easy to use and widely applicable; so, while we have to be able to customise styles, they shouldn’t have to be specified if one wants a plain sheet. I will probably ask my friends for help on the mailing list.

In a totally different direction, I took part in some student things which I can’t really translate in English. One for Physics, about the quantity of information transmitted by measurements (again, translation may have ruined the meaning), and in Mathematics, about the coding of information by neuronal spike trains. The latter was slightly more interesting, but we only had to do a translation — and the professor practically forced it down our throats, but all in all both events were useful and failry exciting (intellectually, mind you).

I also helped RobyC with a very interesting piece of homework: compressing a bmp file into jpeg. Most of the program was already done, including the headers and file input and output, all we had to do was encode the information. This proved difficult because of not having read the homework specification thoroughly enough. We spent hours debugging, with hex editors and all, only to have someone suggest a detail which we had left out. Infinitely stressing, especially since I had an exam the following day, but also very interesting; you kind of get that warm feeling of accomplishment when you see it’s actually a stadards-compliant jpeg file.

Busy as I was, I got into some serious Armagetron Advanced with the boys these days, causing me to see coloured walls in my sleep and to miss this mornings Data Structures course, which I heard was surprisingly interesting. Heading home tomorrow, with that guilty feeling of leaving Roxi behind and skipping two days of school, but also happy I’ll finally get to see my family (6 weeks is apparently quite a lot by my standards). I’ll have a lot of work to do for Numerical Methods and other projects when I return, but I have to get it over with somehow.

Damn anal wordpress added extra line breaks and my text (pasted from vim) looked like shite.

I took advantage of the upcoming exam in Data Structures (I didn’t study for this one) to get some work done on my part of cspay. Libspreadconv is the part that converts a data structure (which I have defined) into an ods spreadsheet. The current implementation is incomplete (even the final version will be incomplete, but this is a subset of the subset) and probably extremely buggy — I can’t read valgrind output, but it does say that a lot of bytes have been lost and this can’t be good. I’m starting to have second thoughts about the way I’ve implemented certain things, so I should probably change them while the library is still not used by anyone. I also want to learn a bit of valgrind and clean up things. Bottom line: the current version is sloppy, slow, buggy and incomplete, but it was quite amazing to run the output through a validator, see it turned out ok, then to actually witness OpenOffice.org display what I intended.

A large part of the code I wrote is actually comments, but I like the way Doxygen spat them out. Right now, however, I have a (repeating myself) small, sloppy library with few routines and clumsy comments. But it’s a start. I think.

I’ve also been playing a game these days — the Romanian equivalent of TribalWars[1]. Sure it’s silly and time-consuming, but right now I _have_ time. When things change (or the game gets harder), I’ll probably quit, like I did with Utopia and the Blue Gecko games.

[1] http://www.triburile.ro/43827.html

Things have been busy lately and I haven’t really been in the mood to blog. A quick recap of what I can remember:

• wrote a small part of World of USO (in Romanian, not open to the public). This was awesome — using C and flex to parse some data and then update a MySQL table. World of USO is educational software integrated with Moodle, but it’s really very closely linked to the “Usage of Operating Systems” course I had last year (approximate translation).
• got my driver’s license. Hate every single moment while driving. Luckily I don’t own or need a car.
• school started. Electronics is a pain, but otherwise things are all interesting. Assembly, Java, algorithm analysis, systems theory and electronics pretty much sums it up. Philosophy is the odd one and it’s nothing like it should be.
• helped Răzvan with organising ACM 2007 Easter Europe contest. Things were insane — interesting people and new challenges.
• trying to learn Haskell. It’s above all other programming languages I’ve seen. Simple, but far from easy.
• starting Hammerfall (working title, partly Romanian, might not be online at the time of this writing). It’s an attempt for a graphic engine based on OpenGL; after finishing it, we pan a game. Things are now very uncertain — we don’t even know what language we will use.
• almost forgot: cspay is fully working; a lot of work from Lucian got us spitting out xls files. Sure, it probably still has bugs, but things are pretty much on track with cspay.
• Rosedu is getting things moving — freshmen joining, new project ideas, a facelift for the website, new forums, Planet Rosedu. The latter is an awesome idea. Be warned that most of the content on Rosedu is in Romanian and we intend to keep it that — only the software will (hopefully) be in English. Rosedu is education-, FLOSS-, and Romania-oriented. The latter speaks for itself.