szbalint's blog

Famous Tennis Player progresses in Famous Tournament

2010-04-28T18:56:51Z

A famous tennis player from Spain, Rafael Nadal remains on course to retain his title in a very famous championship organized in Italy, after sweeping aside another famous tennis player by a decisive victory to reach closer to victory.

The 23-year-old (Rafel Nadal) played better the Other Player in a match that lasted only one hour and 10 minutes.

Nadal performed well in the first, second and sixth part of the game.

The Other Player was mediocre in the fifth part of the game and thus Nadal won the match when it ended.

"It probably wasn't my best match but I did well, I played safe," said Nadal.

Nadal, a very good tennis player, faces Yet Another Player from an East-European country, if he is to win in Italy.

"I was focussed the whole time and I tried to be more aggressive.

"He made more mistakes than usual which helped me a little bit but my serve was working very well." - Natal said.

Note: serve is a very special expression used among tennis specialists to mean a special kind of move.

There were also a bunch of other players, some winning, some losing encounters, some decisively some not.

Does this sound familiar?

No? It should. This is exactly how it feels when the BBC reports on science. It is dumbed down so much that the essence of the news is lost. It is made entirely devoid of any phrases that might be required to understand the topic. It is presented as proclamations from high above, in the style of absolute authority.

The original piece of this sports related news can be found here. The sports journalist doesn't treat the audience as a bunch of 5 year olds. Neither is that common among financial or other kinds of journalists, except for some reason, science journalists.

Take this news article about the debate on hospital mortality rates, that made me angry enough to write this post and to rewrite it after I lost it to a stupid editing system. That article is a showcase of what's wrong with science reporting at a mainstream news organization. It contains all the typical mistakes: expert proclamation from authority but without evidence, a misguided desire to avoid technical terms, reporting on a finding without including the finding, not including any links to the original material and so on...

The article tells us about the two expert sides debating whether hospital mortality rates should be given less or more emphasis in determining the quality of care at individual hospitals. So it turns out a peer-reviewed scientific article in the British Medical Journal argued that mortality rates are a poor measure. The news report reporting on the article doesn't even say WHY.

It is a counterintuitive finding for non-medical personnel. What could be better at measuring a hospital's effectiveness than to measure how many people it saves? It turns out the reason is chiefly due the fact that there is a low signal to noise ratio for preventable deaths in the morbidity rate (around 95% of the total deaths are unpreventable no matter the quality of care). How do I know this? I know this because the report can be located under a minute. It is freely available in full at the BMJ's website. The explanation is of course not so simple as I've here stated, however that is the crux of it. I found the report itself very readable and enlightening. Most adults with basic reading comprehension should have no problem understanding it at all.

Maybe, in 2010, a website writing about a scientific paper could actually link it? I mean, it's not like that the BBC has a "RELATED INTERNET LINKS" section? Oh wait.

Stop treating your readers as children. They're perfectly capable of dealing with specialist words and complex reasoning. You wouldn't leave such things out from a sports or financial article, so why the compulsion to try to dumb science down to meaninglessness?

Trying to dumb the article down in this particular case resulted in an article that is on many levels harder to understand and more incomprehensible than the original peer reviewed article in the British Medical Journal. Stop the whacky science reporting.
]]>

First Budapest.pm meeting in a while...

2009-10-26T13:16:08Z

The essentials:

Time: 2009.10.29 6pm

Venue:
Kaledonia Scottish Pub,
1066 Budapest
Mozsár utca 9.

Map: http://bit.ly/talalkozo

If you are in Budapest, if you're interested in Perl and even prefer to have a few nice beers, just show up, it's that easy :)

Did I mention the excellent beer?

Why change something that works?

2009-09-01T17:37:09Z

In any large deployment sooner or later someone is bound to ask, "why change something that works?".

Sometimes things are a lot simpler than they are assumed to be. In the immortal words of the Joker when asked his proposed solution to a seemingly complex problem, he said: "It's simple. Kill the Batman".

The answer to "why change something that works" is "because you fucking have to!"

At least if you're talking about large scale deployment of software.

The fundamental issue here is that software changes, evolves and doesn't stay static.

The statement "works" is very much relative. You claim it works today within acceptable tolerance, that's all well and dandy. How about tomorrow? Surely the definition of "works" depends on the details and what someone requires of a system changes over time as needs change.

"But my requirements are quite specific and I don't intend to change them!" I hear you saying. The fun part comes from the fact that you're not fully in control of the definition of a working system.
]]>
One overriding reason to do so might be security. There are a lot of security advisories out there that say "this vulnerability affects releases from the past 10 years, upgrade to the latest version x.y.z released 2 hours ago to fix the issue". It doesn't usually say "this vulnerability affects releases from the past 10 years, but we've helpfully provided a point release to every release from the past 10 years that fixes this problem and you can upgrade to". Security advisories don't say this and developers don't do this because it would be a waste of resources to do this.

If you've been labouring under the assumption that things that work shouldn't be upgraded and then suddently hit an overriding reason to upgrade anyway, it usually results in one heck of a firefighting season when the accumulated technical debt is demanded to be repaid with the accumulated interests immediately.

Open Source software exists as a community effort. The open source developers are a bunch of jolly good happy campers who offer you the chance to use their software and come along for the ride, but if you then decide to camp down and happen upon nightfall next to a disreputable part of a forest and get attacked by a bunch of thugs, then it's not the developers fault that you've parted ways.

Remember, a particular piece of software might work for your purposes at a given point in time, but if you want to ensure that it continues to work in the future, you have to remember that the communities and the software environment built up by those communities do not remain unchanged. They move.

The smart choice is to remain within the community and not get caught in the forest at night. By doing that, you're foresaking all the benefits being part of a community can give you.

Good luck trying to get help fixing a problem with a really old piece of software for example. It is likely that the problem is already fixed in a new version, but even if not noone will willingly work on the old version without a substantial motivation to do so. Remaining in one place excludes you from benefitting from new features, bugfixes, performance enhancements, general advice and the ability to pay someone to fix something.

Looking at things this way makes it obvious that the choice isn't between upgrading or not upgrading, but in setting the terms of how to upgrade and when. It can be done as a routine operation, in a planned and dependable way, receiving the benefits of doing so or it can be done in an unplanned and haphazard way when an overriding reason manifests.

The routine part is important aswell, not just to minimize the fuss from an upgrade, but from a software development perspective aswell. Software development is rarely revolutionary, but it is evolutionary. It is an iterative process. There is no magic cure or "oh, I'll just upgrade every 2-3 years to a version that has the features and then I'm safe!" mindset.

For example suppose that you've been running an old kernel, say 2.6.16.x and you're interested in knowing which newer version improved on tcp congestion avoidance algorithms and throughput. In truth, practically all of them. 2.6.19 made CUBIC v1 the default, then CUBIC went through a series of optimizations and revisions to arrive at v2.3 in 2.6.29 as of this writing.

There are of course costs involved in keeping things up to date. Proper procedures have to be established and testing needs to be performed because things can break in an update. However, it is a lot more convenient to have things break during a planned deployment of new software, with the possibility of stepping back than in an unplanned situation with no chance of going back.

Community perspective

On the community side of things and specifically in the Perl development community there is sometimes talk about the DarkPan, the unseen corporate codebase. The argument goes, that deprecating something or dropping support for an old version cannot or shouldn't be done because it might break a large amount of code in the DarkPan.

It is the corporate side of "stability through no change" myth that feeds the DarkPan argument. It follows, that if you accept the argument I've outlined above, then referring to the DarkPan is invalid aswell. Why should a development community hold back on deprecating things and breaking compatibility with very old versions of software, for the sake of future clarity and the sake of up to date users, just because some corporate type out there was irrational or irresponsible enough to get stuck in the past somewhere back along the way. I feel that calling "upgrade freeze" irresponsible is warranted, since keeping backwards compatibility for very old versions of software implicitly means that a community supports software with known security vulnerabilities through not explicitly advising people not to use them!

Perl has, luckily, only a small number of security vulnerabilities compared to other software, but 5.6 still suffers from multiple serious issues from the security perspective. Exerting an effort to keep CPAN modules or features compatible with 5.6 or referring to the DarkPan as a reason not to deprecate/remove some misfeatures from new Perl 5 versions makes no sense.

It is the case of worrying about people having a rough time to upgrade who have shown no indication of wanting to upgrade in the first place, nor having the wisdom of making the upgrade process as smooth as possible for themselves. If inevitable pressures force some of those DarkPan users to upgrade their codebase eventually, they need only to look in the mirror for the cause of any inconvenience they experience.

YAPC::EU 2009!

2009-07-31T21:36:55Z

I'm just done with packing for YAPC::EU, tomorrow I'll take a cozy mid-afternoon flight to Lisbon and then the fun starts ;-)

Hopefully I won't get too carried away and I'll have the opportunity to take some pictures for blog's sake...

Can't wait!

Magyar IRC csatorna Perl témában

2009-07-15T20:03:59Z

Gábor javaslatára ezentúl aki magyar nyelven szeretne Perles és nem Perles témában beszélgetni vagy segítséget kérni, azok számára:

#magyar.pm

A Padre beépített linkgyűjteménye is ide mutat. :-)

Perl, Én Így Szeretlek

2009-06-11T06:43:18Z

Nincs rendszergazdai állásom, nem vagyok netbetyár, és nem rosszul összetákolt shell és cgi szkripteket írok. A világ egyik legszebb és legkifejezőbb nyelvét beszélem, s rajtam kívül még több millióan értik, hogy mire gondolok, amikor azt mondom:

$_="krJhruaesrltre c a cnP,ohet";$_.=$1,print$2while s/(..)(.)//;

A pólómon elfér jópár teljes program, ha konferenciára megyek és szinte már örülök, hogy ha a Javasok összekeverik a sigilt a twigillel. Megvetéssel gondolok a PHPre, a Ruby on Rails sztárolására, és az összes CPAN nélküli programozóra. Nem szeretem a 20+ soros Hello Wordöket, a szigorú típusosságot, a C-t és a .NETet se. A választott nyelvem hajtja a web felét, mégis azt mondják, elavult nyelvet használok. Szövegmanipulációban mi vagyunk a császárok, és igenis nálunk vannak a világ legjobban tesztelt moduljai. Perl, én így szeretlek.

If you have no mouth, you cannot scream - please kill RSS!

2009-06-02T18:00:39Z

A while ago I've started using Liferea (a feed reader) after I've realised that I'm spending way too much time just checking websites for new content. It was time to call it web bankruptcy and switch over to feeds.

Things were good for a while, time passed and more and more feeds were added.

Eventually the Ironman challenge launched and I've subscribed to its Atom feed. All seemed fine until some posts appeared as a html tag soup, tags displayed as text instead of actually used.

This is Perl, I should be okay with that, let's fix that bug then - I thought and that's when the fun started.

]]> Ironman uses Plagger to manage feed aggregation and after a bit of looking around I've found out that it uses XML::Atom internally to publish the Atom feed.

Atom stores the meat in the body of the content element and the WTFing on my side started when i realised what the relevant part of the code does:

Simplified, it's something like:

eval {
    my $parser = XML::LibXML->new;
    my $tree = $parser->parse_string($copy);
    $node = $tree->getDocumentElement;
};
if (!$@ && $node) {
    $elem->appendChild($node);
    $content->type('xhtml');
} else {
    $elem->appendChild(XML::LibXML::Text->new($data));
    $content->type($data =~ /^\s*}

In effect, the code tries to parse the data as xml(!) and adds the content node with type xhtml. If that fails and the data begins with a tag, then it's treated as html otherwise as text.

This is where I've decided that XML::Atom is broken on a design level. Attempting to guess format/encoding is a serious antipattern as far as I'm concerned. The format should be specified, not guessed. What's worse about XML::Atom is that in the current code there is no way to override that guess, to specify the content's type directly.

I've committed some hacks to work around this problem for ironman, however the proper fix would be to replace XML::Atom and XML::Feed in Plagger with a proper solution. A worthy task for the lazyweb, eligible* for a vertical meter of beer if the person is present at YAPC::EU 2009 or possibly some later event.

Returning to the ironfeed, I thought the nasty hacks I've made were the last of the tagsoup problem and I can enjoy a well formatted feed in the future. It turned out not so.

If you have no mouth, you cannot scream

To understand my shock it has to be said that I learn as I go along, so before I started out fixing the content type of the feed entries I didn't know too much about RSS or Atom or the modules used to create the feed.

Imagine my shock when I found out that trying to extract type information from the most commonly used feed format, RSS 2.0, is impossible by design, because that piece of sh not well designed format does not supply one.

Let me reiterate once more: guessing encoding or formatting doesn't work. It belongs in the the hall of antipatterns where people parse html with regular expressions and embed SQL in javascript.

Because of this, I'd like to make a humble request:

Please, please, stop using RSS and publish your feeds using Atom 1.0, so that we know how to interpret the content you're publishing!

Thank you.

* Eligibility is determined by my subjective judgement. A necessary but not sufficient condition would be to be able to generate a valid Atom ironman feed with it with no detectable formatting problems where data is from atom sources and to have tests.

Bilingual Blogging Builds a Better Budapest.pm?

2009-05-27T21:23:07Z

If the english language Perl community needs some publicity, buzz and marketing that's twice as true for hungarian Perl users. So, as suggested by szabgab++, I'll be blogging partly in hungarian to improve upon this unfortunate situation.

]]>
Már egy ideje készültem megírni ezt a bejegyzést de ahogy szokott lenni közbeszólt a technika ördöge és minden egyéb. Szombat reggel arra keltem fel, hogy nem tudom rezolvolni a dev.perl.hu-t és nagy csodálkozásomra gyorsan kiderült, hogy a domain felmondás alatt van.

Valami történhetett a céggel (egy kis cég volt?) ahol hosztoltam a domaint, mert mindenféle értesítés nélkül jutott idág a helyzet. Kis utánajárás után most szerda délutánra sikerült átvinni a domaint egy megbízhatóbb helyre.

Apropó, perl.hu, idő hiányában nem jutottam odáig hogy valamilyen tartalmat is prezentáljon a domain, ezért ha valaki kedvet érez összehozni valamit a közösség számára azt szívesen veszem, jelezzétek nyugodtan...

Látszik, hogy ráfér a változtatás a helyzetre, hiszen minimális szintű aktivitás jelentkezik csak Perl témában itt Budapesten (nagyrészt az egy szem levelezőlistán), például Béccsel összehasonlítva, a magam eszközeivel ezen próbálok is változtatni.

Tavaly ősszel tartottam a Schönherz koliban egy Perl kezdő kurzust, ha minden jól sikerült akkor a slideok fel fognak előbb-utóbb kerülni ide is.

A kurzusnak ősztől tervezem igény szerinti folytatását illetve ismétlését valahol az ELTE-BME-Schönhertz Bermuda háromszögben, tehát minél több visszajelzést kapok ezügyben annál jobb ;)

Addig is, ismeretlen rejtőzködő magyar Perl programozó, blogolj, írjál a levlistára, hallasd a hangod!

Charting Overchoice

2009-05-18T19:09:32Z

Charts are awesome. Humans are excellent at pattern recognition and that makes charts a very good way to convey information.

]]> on the contrary actually.

I think what manifests here is overchoice. There are too many similarly looking modules and the problem space is pretty broad.

A lot of the chart generating modules feel dated. No support for antialiasing, weird fonts and colours and the inability to scale to a larger size (200x200 or 400x300 pixels is simply small) and a larger dataset (more than 20 values on a chart?) are common issues in GD::Graph, Chart, Chart::Strip and various other modules that pop up for a CPAN search. There is no clear leader among the graphing modules.

For most people who want to generate them, charts are helpful visualizations. It isn't an area of focus that succeeds in raising interest for a longer period of time. People search CPAN, find a module that is "good enough", write some quick hacks to get it done or publish yet another alpha stage module, but putting together something really useful requires more than that.

I remember being annoyed about 3 years ago and writing a pie chart generator module based on Imager that did antialiasing and a few other useful tricks, but never getting to release it. I've started working on something else and then lost the code in a harddrive crash. I've never bothered rewriting it again, since like most people I was only tangentially involved with charts.

A couple of weeks ago I wanted to create some charts again, hoping that I don't have to roll my own charting code, I've started searching CPAN again for modules to use and eventually I've settled with SVG::TT::Graph::TimeSeries to create some largeish visualizations.

Using the svg format solved display size related problems and overall I'm pretty happy with the module. If it would include better ways to manage bounds and data labels, it would be excellent for most "fire and forget" uses.

Overchoice applies to development, too. Developers hear the phrase "don't reinvent the wheel" often enough, but if reinventing the wheel isn't preferred, which module should they contribute to from the many available choices?

Perl programmers shouldn't care about memory and CPU usage

2009-05-09T19:02:45Z

There is this old chestnut of wisdom, that Perl programmers shouldn't care about the memory and CPU efficiency of their programs, or to put it differently "if you care about memory and CPU efficiency, use C and write your own garbage collector".

I often wondered about the wisdom of that advice, so one day I've done the opposite and started using Perl for memory, speed and CPU cycle intensive tasks. I don't want to write more C than I can help, I want to use Perl, since I like programming in Perl. So why should I care about speed and memory usage until I have to?

Certain use cases excluded, like embedded programming where memory and CPU power is extremely limited, on mainstream desktop and server hardware it is a good first approximation to start coding in Perl and improve on efficiency as needed. If Perl is your kind of language then avoid the temptation of deciding that Perl can't possibly cut it for high performance environments.
]]>
Talking about memory usage, for my use case initial memory footprint is much less important than the need to maintain a steady, near constant usage of it as I'm dealing with long running processes that do a lot of work.

Fortunately sticking to a few basics, it is much less of a challenge than expected.to achieve exactly just that. Perl uses a reference count based garbage collection, which means that the elephant-in-the-room mistake to watch out for is creating circular references. Reference count based garbage collection reclaims memory when the reference count of a variable reaches zero. If variable A points to B and B points to A, the reference count remains higher than zero and the memory never gets reclaimed, resulting in a memory leak.

A solution is either to avoid creating circular references, remember to explicitly break them or what's even better: use Scalar::Util's weaken, to designate one of the references to be weak, that is to mark the referenced variable free for destruction if only weak references point to it.

It is possible that despite our best efforts some code leaks. There are a variety of ways to deal with that:

A large array of helper modules are available, like Devel::Leak, Devel::TrackObjects, Devel::Size, Devel::Cycle just to mention a few.
If you suspect that perl itself or an XS library leaks memory then a memory profiler like Valgrind might be useful.
Memory leaks can be tricky to find even with all the tools available and personally I found that the best approach in dealing with them is not a tool but a method. Triangulating a memory leak by building a list of observations about your program from as many independent variables as possible and using a bit of logic to deduce how, under what circumstances exactly your program leaks (this is something like doing git bisect, but instead of commits you deal with observations, with the added difficulty of having to sort those observations to see a pattern).

So far I've had more problems dealing with memory leaks* that have arisen from using C based libraries or modules through XS interfaces than from using Perl code, although I did run into the problem that for my use case perl itself, version 5.10.0 proved to be too leaky (5.8.9 is fine though and in bleadperl all leaks I've had problems with are already fixed, so 5.10.1 is shaping up to be a good, stable release from a memory perspective). Perl 5 is written in C, so maybe Perl 6 is on the right track by planning to be self-hosting. :)

On the CPU cycle front avoiding premature optimization is JIT for programmers. Optimize when you have to, have enough resources to do so and when you have context where to start.

Context means information and information comes from profiling. For profilers in Perl there is only one reasonable choice, not because the lack of alternatives, but because Devel::NYTProf is that good. It is fast, versatile, robust and very informative.

By using Devel::NYTProf the information allowed me to write some high performance code that looks pretty interesting when it comes to CPU cycle usage. Most of the cycles (80%+) are spent in well-written C libraries, even though I've written most of the code in Perl, with the exception of contributing some to an XS module. I've got to eat my cake and keep it, too.

So, you like writing code in Perl? Then enjoy life, don't worry and JFDI. It might turn out much better than expected.

* You can always count on Microsoft to provide an implementation of something that happily trods off the beaten path thereby exercising corner cases.