What’s going on with Linux right now

May 8th, 2008 by ripa

Greetings from HP Linux Forum 2008! Some members of our team are participating one of the biggest Linux events here in Helsinki, Finland.

The crowd here is a nice mix of traditional Linux nerds, some people a bit more business-oriented and some from public sector. Linux seems to be doing well - judging from the amount of participants - as the subjective estimate is that there are more people than last year.

Linux Forum 2008

Hot topics are of course still virtualization, just like last year, interoperability in heterogeneous environments and also the trend towards communities. There’s a nice SMS poll system in use here, showing the poll results on a big screen in real time. One poll showed that way over 80% of the participants were using both Linux and Windows in their companies. The Finnish Linux User Group’s prize went to the Ubuntu Finland community. Congratulations, well done!

There was a very good and entertaining speech from Teppo Sulonen, the CIO of City of Tampere. He was praising not only open source software, but openness in bigger sense. Three big cities in Finland have started working together on a common ICT infrastructure. The system is being implemented using common standards and open source technologies instead of the old style of all the cities building their own incompatible systems using proprietary technologies.

Karl Paetzel from HP commented that even if Linux necessarily isn’t in every corner of every datacenter, it is mainstream for almost every customer they talk with. So I think that even if the great coming of desktop Linux may not be here yet, it looks like Linux is very much mainstream in almost every other area.

Signing out,
Ripa

New settings for archive scanning

April 25th, 2008 by sti

Linux Security 7 is now out and with it is a new version of Security Platform, v. 2.0. Security Platform is our scanning core. It contains the scanning daemon and everything else related to malware detection.

The scanning daemon, fsavd, has some new settings. These did not make it to the LS7 manual, mainly because we did not think too many people would be interested in peeking under the hood.

For those who are interested, here are the details:

First setting:
F-Secure Security Platform / Settings / Advanced / Archive Settings / Maximum archive size to decompress into memory (1.3.6.1.4.1.2213.48.1.100.10.10.10)
Any archive smaller than this size will be decompressed in memory while it is scanned. The default value is 50 MB. Valid values for this setting are 1 - 8000 MB

Second setting:
F-Secure Security Platform / Settings / Advanced / Archive Settings / Maximum archive size to decompress into temp file (1.3.6.1.4.1.2213.48.1.100.10.10.20)
Any archive larger than previous setting will be decompressed into a memory mapped temporary file. The default value is 100 MB. Valid values for this setting are 1 - 80000 MB.

These settings allow the user to fine tune the speed of scanning archives. Archive scanning is essentially a function of how much memory can be allocated to the task. Scanning is fastest when the whole archive can be decompressed into memory for scanning. Users can now allow fsavd to take as much memory as they feel comfortable with.

Some archives are so big they will not fit into memory. Those archives will be decompressed into a temporary file, which is mmap’ed in the scanning daemon. The 2nd setting specifies the maximum size for that temporary file.

We do scan archives even larger than can fit into the memory mapped temporary file, but that might be considerably slower because only a part of the archive can be decompressed at a time and might even need to be decompressed again if later analysis requires a part of the file to be re-examined.

In a nutshell: archive scanning is a compromise between speed and size. If you have lots of memory, you can have fast archive scanning. If you do not have a lot of memory but have a lot of disk space, you can have reasonably fast archive scanning. If you have neither, you are going to have slow archive scanning.

Third setting:
F-Secure Security Platform / Settings / Advanced / Archive Settings / Directory for temporary files (1.3.6.1.4.1.2213.48.1.100.10.10.30)
This setting specifies the directory where the memory mapped temporary files are created. The default directory is /tmp. The temporary files are unlinked immediately after they are created, so you will probably never see the files.

If you never want fsavd to create temporary files, set the 2nd setting equal to the 1st setting. Then all archive decompression will happen in memory.

Fourth setting:
F-Secure Security Platform / Settings / Advanced / Archive Settings / Maximum allowed compression ratio (1.3.6.1.4.1.2213.48.1.100.10.10.50)
Some archives do not contain real files but are maliciously constructed to cause havoc in an AV scanner by blowing up to an extremely large size. This setting allows fsavd to protect itself by issuing a scanning error for archives which have very large compression ratio. The default maximum compression ratio (decompressed size / compressed size) is 1000. Valid values for this setting are 1 - 1000.

Linux Security 7.00

April 21st, 2008 by hessu

Today we are proud to present a major piece of quality software, which we have improved, tested and polished with all of our skill, understanding, passion and love for the past 11 months. The release of Linux Security 7.00 has arrived. I’d like to highlight some of the improvements here:

  • New web-based wizards have been added for manual scanning and configuring the integrity checking and rootkit protection features.
  • A simplified, cleaned-up installer with less questions asked.
  • A new kernel-level scanning result cache significantly improves the performance of on-access scanning.
  • The new F-Secure Scanning Engine has been integrated.
  • We added methods to completely disable specific components of the product, like the firewall or the web user interface. If you prefer using your distribution’s firewall configuration method, we will not interfere with it. If you prefer to not use the web user interface, there is no need to burden your system with it’s Java run-time environment.
  • Firewall rules can now be applied to specific network interfaces.
  • The F-Secure Gnome panel applet now notifies you of any security alerts.
  • The client and server edition installer packages have been merged into a single installer to simplify distribution and installation. On the other hand, new 64-bit installer packages have been introduced to fully support new 64-bit distributions. Some things simply wouldn’t work with 32-bit compatibility libraries.
  • If you’re still running F-Secure Antivirus for Linux 4.65, it’s now possible to upgrade to Linux Security 7.00 in command-line-only mode. If command-line scanning is all you need, this is for you.
  • If you wish to integrate F-Secure’s cutting-edge scanning features with your own software, the Linux Security 7.00 release package contains an SDK for our daemon API, full with header files, a manual page, and example code in the C language.

With a large number of new Linux distributions added and a couple of old ones removed, this release is officially tested and supported on 33 different distribution versions. It should work on a few others, too, especially if installed in command-line-only mode.

Unfortunately the native Gnome scanning application, fsgav, had to be dropped before the release. Don’t worry, it will probably re-appear in a future version, once we’ve had time to smoothen it’s sharp edges a bit.

A full list of new features, changes and supported platforms can be found in the release notes. Please download your evaluation copy here:

  • >-secure-linux-security-7.00.71615.tgz (MD5, SHA1)
  • f-secure-linux-security-64bit-7.00.71615.tgz (MD5, SHA1)

As usual, the software can be evaluated for free for 30 days.

UPDATE: this build has been recalled. More information will be available here shortly.

If you wish to purchase licenses for your business, please get in touch with one of our sales partners or regional offices. End-user licenses should be available in the F-Secure eStore in the near future.

Stuff That Works part 4: Continuous Integration

April 14th, 2008 by sti

We have been very busy recently (aren’t we always?) because we are preparing the final release of Linux Security 7. Even though we try our best to be agile and use Scrum, things still seem to speed up towards the end when every string is tied and every surface polished. We are excited about this release. Linux Security 7 has a number of new features and fixes we are sure you will like.

Last week I wrote about the importance of one command build. Among other benefits, it makes it easy to automate the build. And when you have an automated build, you can run the build as often as you want.

Why would you want to build often? Let me tell you…

Five years ago I worked in a team where we had a weekly build on Thursday afternoon. Every developer worked on their own module from Monday to Thursday morning and then did cvs commit. Then the Build Master made the build and invariably it failed to compile. Then the Build Master figured which developer to blame for the compilation failure and the poor developer tried to frantically make his code co-exist with all the changes others had done. When the developer committed his fix, the Build Master tried to build again. Lather, rinse, repeat.

When the Build Master finally got a build that compiled, he tried to install it. If it failed to install, we digged into it to find out why, fixed it and started over.

Sometimes it took until Friday afternoon before we had a build good enough for Quality Engineers to start testing. Needless to say, you could not commit anything during building or you could risk breaking the build.

This was a very stressful time! I really began to hate Thursdays.

When the next project started, I wanted the team to start building every night. People were generally against this saying it made no sense to build every night because the Quality Engineers could only test one build per week. Eventually I got permission to make an automated build script and make builds every night. (We agreed the Quality Engineers will test the Thursday build and pretend the other builds do not exist.)

It was much better! The nightly build revealed build problems earlier, every night really. The team no longer had to wait until Thursday afternoon to find out their source tree does not compile. Every morning the developers had email telling them if the build failed and it was pretty easy to see who should fix what. I was not stressed any more.

The Linux team has had automated nightly (well, daily, because we build at 4 pm) build from the start and we also do automated hourly builds.

An hourly build is even better than a daily build. You find out problems with your source every hour! This means you can fix them much sooner. And when you fix problems as early as possible, your daily build will be in much better shape. In fact, I don’t even remember the last time we had a broken daily build.

Why not take this even further? Why not build every 15 minutes? On every commit?

Currently we only build hourly because we test every hourly build. The hourly test is fairly comprehensive, it includes installation, basic functionality, some detection tests, even some WebUI tests and an uninstall at the end. All this testing takes some 40 minutes.

After Linux Security 7 is released, we plan to spread the test on multiple machines to make it faster. Hopefully we can then start building+testing every 15 minutes or less.

The holy grail of Continuous Integration would be to get immediate feedback when you do a bad commit. I’m not sure how we get to “immediate” but we keep on improving and that is what counts.

Stuff That Works part 3: Silent Hour

April 4th, 2008 by Kati

For a smallish team, communication and teamwork is usually smoothest when sitting in the same room. But, as the team grows, every person adds to the general noise and fuss levels in the room. People don’t just bring the noises they themselves make into the room. Every person also brings in more phone calls, visitors and spontaneous discussions.

We currently have 11 people sitting in the same room and I have to admit that at times our working environment is pretty hectic. We don’t get a lot of phone traffic, but it’s quite common to have a few conversations going on simultaneously accompanied by sound effects that state our latest hourly automatic test results. Luckily most of the team members are not easily disturbed, and don’t seem to be having concentration problems too frequently. The issue does come up at times though, and there has been one major “innovation” that has helped us a lot - Silent Hour.

Our daily scrum meeting is at 11.00 every day. After that there usually are some more discussions going on for a while, and then we go for lunch. Later, in the early afternoon, generally the most productive time of the day starts. To make sure it is used to the fullest, we have our Silent Hour (which actually lasts for 2 hours). We ourselves shut up (pair work with low-volume discussion is ok) and move any conversations to our IRC channel or out of the room, into the kitchen or a meeting room. A STOP sign on the door stops wannabe-visitors and asks them to lure whoever they want to talk to out of the room. Phones go silent as well.

We’ve had Silent Hours since last summer. At first we were afraid that our visitors might not appreciate it, but generally there have been no issues. Of course, every now and then someone just marches into the room to state their business and needs to be guided out, but surprisingly there have been no complaints over that. Actually, one co-worker thanked us for having the Silent Hour info on the door because it implicitly says that it’s OK to visit us outside the Silent Hour.

Sometimes we have forgotten to be silent ourselves. Still, the value of Silent Hour is verified by the fact that it hasn’t been forgotten for more than one day at a time.

Stuff That Works part 2: One Command Build

March 27th, 2008 by sti

Last week I wrote how important it is you can set up your build environment by running just one command.

Well, perhaps it is a worthy target to aim for, even if you miss it slightly. I showed you our build environment set up:

$ wget -O- http://server/1cmdenv.sh | sh
$ cd universe
$ make client-server-5.5x

Strictly speaking, that’s 3 commands (wget, cd, make). I’ll argue that cd is not really a “command”, as such, and since we have so many source trees you need 2 commands: 1 to check out the “tree-of-trees” or “universe” as we call it, and 1 to check out the source tree you want to build. I would say that is as close to one command build environment setup as we can get.

The other alternative would be to create separate setup commands for each source tree. Then where do you store these commands? Probably in the version control. But then you need another command to check them out. So, 2 commands is reasonably close to the ultimate goal.

Now to the subject of this post: The one command build.

We have chosen the one command to be: make. Running just “make” with no parameters performs a full build. With full build I mean it produces the installation package that is shipped on cdroms and uploaded to web sites.

Whatever the command is, it is important it does everything with no manual tweaking.

If there are manual steps in your build procedure, someone, somewhere, sometime when you least expect it, will get them wrong. Script it. All the way.

Even when your build is just one command, someone will think he is in a hurry and he doesn’t have time to run the one command. He will try to cut corners. He will think: “I already have module 1 built on my workstation. I’ll use that and build modules 2 and 3 by hand and execute the packaging command from the temp directory. I’ll save 10 minutes.” Beware! That is the way dragons lie.

What’s wrong with some shortcuts? Nothing if you can be absolutely sure you know what you are doing. Can you? Do you trust yourself? You do remember what causes most accidents in nuclear power plants? Human error.

4 months ago one man at the waterworks of Nokia, Finland (the city, not the phone maker) accidentally left one valve open and as a result contaminated the drinking water of tens of thousands of people. It took months to clean up the water pipes.

Long, long time ago I attended a lecture on fault tolerant systems. The lecturer said: “If you only remember one thing about this lecture, let it be this: The easiest way to build a fault tolerant system is to place the system in a very strong cabinet, lock the door and lose the key.”

In other words, eliminate the human factor.

When we run “make” in our source tree, it will do at least these things:

  • generate dependencies
  • export header files and string tables from a MIB file (contains all the settings and alerts of a software product in the F-Secure Policy Manager system)
  • run a Perl script to generate code from the headers exported in previous step
  • run another Perl script to verify string tables above match the localized string tables
  • compile and install the unit test framework
  • compile and run unit tests
  • run a few static analyzers
  • compile everything that is needed
  • create RPM and DEB packages from the compiled code
  • create a self-extracting installer from the RPM packages and an APT repository from the DEB packages
  • export system tests from the source tree to be executed on the build by an automatic test system
  • copy all the stuff that was generated to a separate file server

I’m sure I forgot a few steps in the above list. There’s too many to remember. The beauty of it is: I don’t have to. The one build command does it all for me.

When the build is just one command, it is easy to automate it. When the build just happens automatically, it further removes any possibility to screw it up. Create a cron job to make the build every night.

When you have a cron job making the build every night, why not make it every hour? Or every 15 minutes? When you have that, why would you bother making a build by hand ever again if a cron job is going to do it anyway in a few minutes. And it will do it better than you. Always the same way.

That sounds like Continuous Integration. Maybe I’ll write about that next week.

New security updates

March 20th, 2008 by Rasmus

Just to make sure the message goes out: as mentioned by our colleagues in the lab, this week we released an important security advisory “FSC-2008-2″, about potential vulnerabilities affecting components common for many of our products, including Linux products. If you’re running one of our Linux products, chances are that you are affected and you should upgrade to the latest build immediately. Read the advisory for more information, or get more details from CERT-FI.

To get e-mail notifications about maintenance releases, subscribe to our maintenance release notification mailing list.

Stuff That Works part 1: One Command Build Environment

March 19th, 2008 by sti

We are going to write a series of posts on good practices concerning software development, configuration management, building, testing and methodologies. The stuff in these posts will not be vague, abstract ideas but real lessons learned by veterans in the trenches of the battlefield.

This is the first post in the series, so let’s start from the beginning.

Everyone of us has been in this situation: You are the new guy in the team. You have just set up your computer. Email and web browser works. You are eager and ready to start coding!

What is the first thing you must do to start coding? You must install some tools and libraries. You must find out where to get the source code. Which version control tool do they use here? Where is the repository? Who do I talk to get access?

Do you remember how long it took before you were able to build the source code you were supposed to be working on? And build it in such a way that it really resulted in something that installs and works?

It probably took you anywhere from a few hours to a few days (and I have read about horror stories where it took weeks.)

Why?

Because the old hands in the team already have their own build environments set up and have had them for years. Their workstation has all the tools, all the libraries and all the little undocumented tweaks that are needed to make the perfect build. The build environment has not been set up, it has more like grown or evolved. Documentation, if any, is probably outdated.

The only way to avoid this mess is to make it possible to set up a complete build environment by running one (1) command in a clean workstation.

Why one command? If there are two commands, you might not remember them both or you might run them in the wrong order. If there are more commands, you definitely will not remember all of them.

And why would you bother with anything more than 1 command? If you currently need to run 2 commands, make a script that runs them. Now you have only one command. You have just automated the whole set up process and reduced the number of things you need to remember about it by 50%!

What this command does is entirely up to you. It can install software from network drives, copy files, check out source code, whatever is needed to turn a freshly installed Windows or Linux or Mac into something you could replace your build server with.

If you have such a command, it is probably a script. You do not need documentation, just read the script. If there is some obscure stuff in the script, write a comment into the script.

Test the script. Get your boss to buy you a new, shiny, fast workstation and use the script to set up your build environment on it. (If your boss refuses, set up a fresh virtual machine or something.)

And best of all, if one day you see smoke coming out of your build server, you know how to make a new server in no time at all.

We in the Linux team have a script that uses apt-get to install bazaar, configures it, then checks out an Arch project that has a Makefile. This Makefile is used to check out the various source trees we work with. The Makefile also knows which tools you need for which source tree and will install the tools before checking out the source.

This is how it would look if you wanted to make a build in a clean computer:

# One command build environment setup
$ wget -O- http://server/1cmdenv.sh | sh
$ cd universe
# Check out one source tree and install the tools it needs
$ make client-server-5.5x
$ cd client-server-5.5x
$ make testbuild
# Now install and test the build in ./release directory

Of course, it may not be a good idea to depend on tools and libraries that are part of your OS. When you upgrade the OS, you sometimes get new versions of them that are not compatible with the way you use them. But that is a topic for another post.

Fresh from the oven: F-Secure Linux Client/Server Security 5.54

March 7th, 2008 by Tuukka

We have just released a new version of F-Secure Linux Client & Server Security. This 5.54 is mainly a bug fix release, including a new Automatic Update Agent and making the Web User Interface work properly with Microsoft Internet Explorer 7. Please check the release notes for more information on what has changed. As always, we recommend upgrading to the latest version even if any of the fixed issues do not seem to affect your environment. Please feel free to ask if you have any questions regarding this upgrade, and do drop us a note telling how the upgrade went :) The email address is at the bottom of this page.

The product packages and release notes are available from our Webclub:
F-Secure Linux Client Security 5.54
F-Secure Linux Server Security 5.54

Adapting and the importance of feedback

February 19th, 2008 by Kati

As you may have read between the lines of our blog posts so far, we’re an agile team. We use Scrum to do our thing. In practice this means that instead of making rigid plans for a long time ahead and then following them, we split the work into iterations and try to adapt quickly as we learn more and the customer and market needs change.

This is how it works. We split our release projects into (usually) one month sprints. In the beginning of each sprint, we select the most important things to work on next and do detailed planning only for them. As you have probably noticed, we also strive to have some new features ready and released every sprint. The main idea here is to get feedback for the things we do as quickly as possible so that we can learn and adapt accordingly.

The point of this post is just to make sure you know we really (yes, rly :)) appreciate every little piece of the feedback you have sent us so far and hope you keep it up!