The first front-end at which you will stick is dselect, which is a menu based program.

I will not be extended with it because it is simple. What you need to know is that you can update the list of available packages with update, you can select which of them you will install with select. In this option, you go out of the help (going out of the help is the more tricky part) with the space bar (ubuntu like i thing it is with “enter”).

You can look for a package named for example firebird tipping “/fire” (without quote), and it will look for the first in the list named fire-something package and typing “n” you go through this list. Of course it is better typping “firebird”. If you want to select for installing you tip “+” (“-” for deinstalling) and dselect present you the list of conflicting, recommended … packages that you can choose with the same +- method (you go out of the list by typing enter). Once you’re satisfied, you tip “enter” and it go through the install process.

dselect is easy, not always the more flexible but the more secure and stable you can dream about. But let me present you the second soft of the debian classical tools collection: DPKG.

dpkg is a command line based tool. You can save the list of the installed packages on your system with:

dpkg –get-selections > selections.dpkg

Don’t be so shy and edit this text file. If you want to recover the state of your package selections you type:

dpkg –set-selections (no description available)
pn lynx-cur-wrapp (no description available)
un lynx-ssl (no description available)

and if you wish to know all the files that install or simply modify the installation of a package:

dpkg -L lynx

/.
/usr
/usr/bin
/usr/bin/lynx.stable
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/lynx.1.gz
…
/etc
/etc/lynx.cfg
…

You want the information on a package (lynx for example)?

dpkg -p lynx

Package: lynx
Priority: optional
Section: web
Installed-Size: 4292
Maintainer: James Troup
Architecture: i386
…
Description: Text-mode WWW Browser
Lynx is a fully-featured World Wide Web (WWW) client for users
running cursor-addressable, character-cell display devices (e.g.,
…

That is what a look at /var/lib/dpkg/available will give you after a painful search.

The status of the package can be found with:

dpkg -s lynx

Imagine that you find a strange file in you system (bad news…), and you want to make sure which package it comes from:

dpkg -S /etc/lynx.cfg

lynx: /etc/lynx.cfg

And you can run again the pre-installation process thanks to:

dpkg-reconfigure xserver-xfree86

It will replay the pre/post install script at wich you can take a look at /var/lib/dpkg/info/. For example, we had in my company a problem with bacula director’s instalation because there was a mistake in /var/lib/dpkg/info/bacula-director-mysql.postinst. We had this faultly line:

MYSQL_PSWD_STRING=”-p”$MYSQL_ROOT_PASSWORD”"

So the given root password was never the one bacula use to connect to mysql as root, we had to transform in:

MYSQL_PSWD_STRING=”-p $MYSQL_ROOT_PASSWORD”

(without the escaped quote)

Of course we have tried to reach the packager. But that’s not all folks. Now comes the bestpart: <i>apt</i>. You do not have to download the lynx_2.8.5-sarge1_i386.deb and after install with dpkg -i. No! Just lauch your branded 16k modem to become the line up and type:

apt-get install lynx

You have heard about a new security update, retrieve it with…

apt-get update

Same effect as…

dselect upgrade

…with:

apt-get upgrade

If you wish to look into the code:

apt-get source lynx

And all the magic stuff is controlled through /etc/apt/sources.list.

deb file:///cdrom/ sarge main
deb http://ftp.de.debian.org/debian/ sarge main non-free contrib
deb-src http://ftp.de.debian.org/debian/ sarge main non-free contrib
deb http://security.debian.org/ sarge/updates main contrib non-free

Or better with a mirror:

deb http://ftp.rediris.es/debian/ stable main non-free contrib
deb-src http://ftp.rediris.es/debian/ stable main non-free contrib

Or the best, an apt-proxy which proxies your downloads (karkoma, will you write an apt-proxy article, don’t you?):

deb http://aptserv.dmz:9999/debian/ sarge main non-free contrib
deb-src http://aptserv.dmz:9999/debian sarge main non-free contrib
deb http://aptserv.dmz:9999/debian-volatile sarge/volatile main
deb http://aptserv.dmz:9999/security/ sarge/updates main contrib non-free

Note the debian-volatile: this is for the highly upgraded packages like clamav, dcc,…

Another file that can be quite useful is /etc/apt/apt.conf. For example if you have an http proxy for going out in internet you will add in this file:

Acquire::http::Proxy “http://proxy.yourcompany:3128″;;

Or you can simply set the http_proxy=http://proxy.yourcompany:3128 in your environment). In the precedent case of the apt-proxy, if you have an http_proxy and of course you don’t want to proxy your apt-proxy you can control it by adding in this file:

Acquire
{
http
{
Proxy::no_proxy=comm.dmz;
}
}

Or add no_proxy=http://aptserv.dmz:9999 in your environment). Note that you have two syntaxes (the first one is the old one and the second one is the new). Don’t be confused with upgrade and update: update is when the source-list has been changed to synchronize the list of the mirrors and upgrade is to synchronize the list of the new availables packages.

One good tool in case of problems with apt is apt-config. dump> will dump for you your configuration with all the parameters (the defaults too):

$ apt-config dump
APT “”;
APT::Architecture “i386″;
APT::Build-Essential “”;
…
DPkg::Pre-Install-Pkgs:: “/usr/sbin/dpkg-preconfigure –apt || true”;
Acquire “”;
Acquire::http “”;

Caution! When you upgrade to a new distribution of debian (it occurs once every two years more or less, that could be the only criticism of debian), change your source-list, perform an apt-get update, an apt-get upgrade and after that an apt-get dist-upgrade.

One last word. Comes aptitude, a new tool (there is a lot of graphical tools that i don’t know…). but for sarge it seems to me that the old ones do a better works (i must take a closer look…).

Enjoy Debian without moderation…

This article will show you how to implement a clustered, highly available and inexpensive solution based on GNU/Linux and combining MySQL as the database engine and Heartbeat as the failover mechanism. The configuration will consist of a 2-node active/passive cluster.

I assume you have MySQL up and running on both nodes and that your are working with MySQL 4.0.13 or above. If not, please refer to MySQL manual here and download a recent copy here.

How does replication works in MySQL

Replication in MySQL is very simple: one machine acts as the master server and one or more machines act as the backup servers (the replica servers). The master server keeps all changes made to its databases in binary log files, so the backup server(s) can read these files and apply the changes to its own copy of the data.

In more detail, the binary log file records all the changes (UPDATE, DELETE, INSERT…) made to the master’s databases since the first time the replication was configured and started. The master also creates and maintains an index file to keep track of the binary logs created. Upon connecting, the slave server(s) obtains new updates from the binary log and aplies them to its copy of the data.

Note: As MySQL suggests, visit their website often to check the latest changes and improvements to its database replication implementation.

How does Heartbeat works

Heartbeat is a piece of software that provides High Availability features such as monitoring the availability of the machines in the cluster, transferring the virtual IPs (more on this later) in case of failures and starting and stopping services.

The Heartbeat software running on the slave server periodically checks the health of the master server by listening to its heartbeats sent via null modem cable and/or a crossover ethernet cable. Note that in the best scenario slave’s main task is nothing but to monitor the health of its master. In case of a crash the slave will not receive the heartbeats from the master and then it will take over the virtual IPs and the services offered by the master.

The overall picture

Next figure shows the picture of our cluster.

As previously stated, our configuration will consist of a 2-node active/passive cluster: dbserv1, the master server and dbserv2, the slave server. Both machines are linked via serial COM port /dev/ttyS0 (null modem cable) and a crossover ethernet cable (eth0), through which they send its heartbeats to each other.

The 192.168.1.103 IP address at eth1:0 is the floating IP address, the virtual IP. This is the service IP where the master listens to and that will be transferred to the slave in case of a failure in the master. Requests from the application servers will be made through the virtual IP.

Both servers have another IP address that can be used to administer the machines: 192.168.1.101 and 192.168.1.102. Bear in mind that the virtual IP (192.168.1.103) is set up by Heartbeat, meaning that if it is not up and running in the active server there will be no access to the virtual service.

Setting up replication

1. Create a replication user on the master:

mysql -u root -p

At MySQL prompt type:

GRANT REPLICATION SLAVE ON *.* TO replica@”%” IDENTIFIED BY ‘replica_passwd’;

2. Stop MySQL on both the master server and the slave server. Take a snapshot of your databases from the master.

/etc/init.d/mysql stop
tar cvzf mysqldb.tgz /path/to/your/databases

In my configuration I would…

/etc/init.d/mysql stop
tar cvzf mysqldb.tgz /var/mysql-data/*

3. Copy the data to the slave

scp /path/to/mysqldb.tgz admin@dbserv2:/path/to/your/databases

If you are using InnoDB tables, copy your tablespace file(s) and associated log files to the slave. In my case, the tablespace is called ibdata and the log files are those ib_*. So:

scp /var/mysql-data/ibdata admin@dbserv2:/var/mysql-data
scp /var/log/mysql/ib_* admin@dbserv2:/var/log/mysql

4. Activate the binary log and assign a unique ID to the master:

vi /etc/my.cnf

Then add/change the following

[mysqld]
…..
# Enable binary logs. Path to bin log is optional
log-bin=/var/log/mysql/dbserv1
# If the binary log exceeds 10M, rotate the logs
max_binlog_size=10M
# Set master server ID
server-id=1
…..

Now you can start mysqld on the master. Watch the logs to see if there are problems.

/etc/init.d/mysql start

5. Log in on the slave.

vi /etc/my.cnf

Then add/change the following:

server-id=2
# This is eth0. Take a look at figure 1
master-host=192.168.100.1
master-user=replica
master-password=replica_passwd
# Port that master server is listening to
master-port=3306
# Number of seconds before retrying to connect to master. Defaults to 60 secs
#master-connect-retry

6. Uncompress the databases

cd /path/to/your/databases
tar xvzf mysqldb.tgz

chown -R mysql.mysql /path/to/your/databases

Make sure your tablespace file(s) and associated files are in place (/path/to/your/databases in our example).

7. Start mysqld on the slave. Watch the logs to see if there are problems.

/etc/init.d/mysql start

8. Check if replication is working. For example, log in on the master, create a database and see if it is replicated on the slave:

mysql -u root -p

create database replica_test;
show databases;


+----------------+
| Database |
+----------------+
| replica_test |
| mysql |
| test |
| tmp |
+----------------+

Log in on the slave server and make sure the database replica_test is created:

mysql -u root -p
show databases;


+----------------+
| Database |
+----------------+
| replica_test |
| mysql |
| test |
| tmp |
+----------------+

If you have problems, please refer to MySQL manual here.

Installing and setting up Heartbeat

Download a recent copy of Heartbeat from here and then as usual….

configure
make
make install

or:

rpm -Uhv heartbeat-1.0.4-1.i386.rpm

if you downloaded the RPM based package.

Configuring heartbeat

There are three files involved in the configuration of heartbeat:

• ha.cf: the main configuration file that describes the machines involved and how they behave.
• haresources: this configuration file specifies virtual IP (VIP) and services handled by heartbeat.
• authkeys: specifies authentication keys for the servers.

Sample /etc/ha.d/ha.cf

# Time between heartbeats in seconds
keepalive 1
# Node is pronounced dead after 15 seconds
deadtime 15
# Prevents the master node from re-acquiring cluster resources after a failover
nice_failback on
# Device for serial heartbeat
serial /dev/ttyS0
# Speed at which to run the serial line (bps)
baud 19200
# Port for udp (default)
udpport 694
# Use a udp heartbeat over the eth0 interface
udp eth0

debugfile /var/log/ha/ha.debug
logfile /var/log/ha/ha.log

# First node of the cluster (must be uname -a)
node dbserv1
# Second node of the cluster (must be uname -a)
node dbserv2

Sample /etc/ha.d/haresources

dbserv1 Ipaddress::192.168.1.103::eth1

This tells Heartbeat to set up 192.168.1.103 as the virtual IP (VIP). See figure above.

Sample /etc/ha.d/authkeys

auth 1
1 crc
2 sha1 HI!
3 md5 Hello!

This file determines the authentication keys. Must be mode 600. As I assume that our network is relatively secure I configure crc as the authentication method. There is also md5 and sha1 available.

Now start heartbeat on dbserv1 and the on dbserv2, watch the logs, then stop heartbeat on the first node and see what happens on the second node. Start again heartbeat on the first node and stop it on the second and see the logs. If all is okay, you have a 2-node cluster up and running.

What we have

At this point we have a 2-node cluster with certain degree of availability and fault tolerance. Despite this could be a valid solution for non-critical environments, in really critical environments this configuration should be improved.

Advantages

• The cluster is fault tolerant
• The cluster is relatively secure
• There is no single point of failure (comments?)
• Automatic fail over mechanism
• Proven and solid OpenSource software for production environment (my experience)
• Simple and easy to install and configure
• Easy to administer
• Inexpensive

Disadvantages

Our cluster presents almost one serious problem in critical environments (i.e. 99,99% availability). As you know, when the master node fails, the standby node takes over the service and the virtual IP address. In this scenario, when the master comes back online again, it will act as the stand-by node (remember nice_failback on from /etc/ha.d/ha.cf?). As our configuration has not implemented a two-way replication mechanism, the actual master is not generating binary logs and the actual slave is not configured to act as such. There are means to avoid this disadvantage, but this is your homework . Let me know your progress.

As usual, comments are very welcome.

References:

• Mysql
• Heartbeat
• More links

NTP (Network Time Protocol) provides accurate and syncronised time across the Internet. This introductory article will try to show you how to use NTP to control and synchronize your system clock.

First approach

NTP is organised in a hierarchical client-server model. In the top of this hierarchy there are a small number of machines known as reference clocks. A reference clock is known as stratum 0 and is typically a cesium clock or a Global Positioning System (GPS) that receives time from satellites. Attached to these machines there are the so-called stratum 1 servers (that is, stratum 0 clients), which are the top level time servers available to the Internet, that is, they are the best NTP servers available.

Note: in the NTP lingo measure for synchronization distance is termed as stratum: the number of steps that a system lies from a primary time source.

Following this hierarchy, the next level in the structure are the stratum 2 servers which in turn are the clients for stratum 1 servers. The lowest level of the hierarchy is made up by stratum 16 servers. Generally speaking, every server syncronized with a stratum n server is termed as being at stratum n+1 level. So, there are a few stratum 1 servers which are referenced by stratum 2 servers, wich in turn are refenced by stratum 3 servers, which are referenced by stratum 4 and so on.

NTP servers operating in the same stratum may be associated with others in a peer to peer basis, so they may decide who has the higher quality of time and then can synchronise to the most accurate.

In addition to the client-server model and the peer to peer model, a server may broadcast time to a broadcast or multicast IP addresses and clients may be configured to synchronise to these broadcast time signals.

So, at this point we know that NTP clients can operate with NTP servers in three ways:

• in a client-server basis
• in a peer to peer mode
• sending the time using broadcast/multicast

How does it work

Whenever ntpd starts it checks its configuration file (/etc/ntp.conf) to determine syncronization sources, authentication options, monitoring options, access control and other operating options. It also checks the frequency file (/etc/ntp/drift) that contains the latest estimate of clock frequency error. If specified, it will also look for a file containing the authentication keys (/etc/ntp/keys).

Note that the path and/or name of these configuration files may vary in your system. Check the -c command line option.

Once the NTP daemon is up and running, it will operate by exchanging packets (time and sanity check exchanges) with its configured servers at poll intervals and its behaviour will depend on the delay between the local time and its reference servers. Basically, the process starts when the NTP client sends a packet containing its timestamp to a server. When the server receives such a packet, it will in turn store its own timestamp and a transmit timestamp into the packet and send it back to the client. When the client receives the packet it will log its receipt time in order to estimate the travelling time of the packet.

The packet exchange takes place until a NTP server is accepted as a synchronization source, which take about five minutes. The NTP daemon tries to adjust the clock in small steps and will continue until the client gets the accurate time. If the delay between both the server and client is big enough the daemon will terminate and you will need to adjust the time manually and start the daemon again.
Sample ntp.conf configuration file

server 134.214.100.6
server pool.ntp.org

peer 192.168.100.125
peer 192.168.100.126
peer 192.168.100.127

driftfile /etc/ntp/drift
#multicastclient # listen on default 224.0.1.1
#broadcastdelay 0.008

authenticate no

#keys /etc/ntp/keys
#trustedkey 65535
#requestkey 65535
#controlkey 65535

# by default ignore all ntp packets
restrict 0.0.0.0 mask 0.0.0.0 ignore

# allow localhost
restrict 127.0.0.1 mask 255.255.255.255

# accept packets from…
restrict 192.168.100.125 mask 255.255.255.255
restrict 192.168.100.126 mask 255.255.255.255
restrict 192.168.100.127 mask 255.255.255.255

Take a look at references below to understand the configuration options.

References:

• NTP homepage
• ntpd
• Network time protocol (version 3) specification
• Public NTP Time Servers

Understanding SSL and TLS

WebAdmins tend to transmit sensitive information safely and securely. To protect yourself from threats like eavesdropping, tampering and impersonation OpenSSL provides mechanisms to encrypt, check message integrity and authenticate your sensitive information.

The communication between both ends starts by the so called handshake process. The handshake allows the server to authenticate itself to the client using public-key techniques, then allows the client and the server to cooperate in the creation of symmetric keys used for rapid encryption, decryption, and tamper detection during the session that follows. Optionally, the handshake also allows the client to authenticate itself to the server.

The process starts when the client sends its SSL version number, random data, etc and a list of cipher suites it supports to the Apache webserver. The cipher suite is a set of cryptographic algorithms that will be used for authentication and session encryption. Apache then sends back its SSL version and other data and chooses a cipher suite (the strongest enabled cipher suites they have in common) from the list sent by the client, notifies to client the choosen cipher suite and sends its certificate (which contains the server’s public key).

Upon receiving the server’s certificate, the client checks it and tries to authenticate the server.

• Checks the certificate’s validity period.
• The client maintains a list of trusted Certificate Authorities (CA) certificates (the root certificate for the key that was used to sign the server’s certificate) that determines which server certificates the client will accept. To verify the certificate the client checks if the issuing CA is in the list of trusted CAs. More specifically, does the distinguished name (DN) of the issuing CA match the DN of a CA on the client’s list of trusted CAs?
• From its list of trusted CAs, the client compares the public key from the CA’s certificate to the one being presented in the server’s certificate.
• Check if the domain name in the certificate match the domain name of the server itself.

If all the above cuestions are succesfully resolved the server is authenticated and the client proceeds with the SSL handshake.

Once all is okay, the client creates and sends a premaster key (which is used as the basis of the session key) encrypted with the server’s public key.

If the Apache webserver is configured to request client authentication (optional in the handshake), the client sends the server both a certificate and a separate piece of signed data to authenticate itself. Now Apache will try to authenticate client’s identity:

• Apache checks if the client’s digital signature could be validated with the public key in the certificate.
• Checks the certificate’s validity period.
• As in the client side, Apache maintains a list of trusted Certificate Authorities (CA) certificates that determines which client certificates the server will accept. To verify the certificate, Apache checks if the issuing CA is in the list of trusted CAs.
• Is the client authorized to access this resource?

If all the above cuestions are succesfully resolved the client is authenticated and allowed to continue.

At this point Apache decrypts the premaster key with its private key and generates a master key which is sent to the client. Both the client and the server use the master key to generate the session keys. The interesting point here is having both the client and the server agree on a session key without ever sending it as plaintext over the network (this depends on the cipher suite selected). Now both ends inform each other that future messages will be encrypted with the session key and that the handshake is finished. This is the end of the SSL handshake and the start of the SSL session.

Configuring Apache to support SSL/TLS

Now I will show you how to configure your Apache webserver with SSL support. I assume you have Apache correctly configured and compiled with SSL support. If not, please take a look at References below.

Edit your httpd.conf and add the following if necessary or modify it accordingly to your needs:

…..
LoadModule ssl_module extramodules/libssl.so
…..

AddModule mod_ssl.c
…..
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl
…..
…..
Listen 443
SSLPassPhraseDialog builtin
SSLSessionCache dbm:/var/log/httpd/ssl_scache
SSLSessionCacheTimeout 300

SSLMutex sem
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin

SSLLog /var/log/httpd/ssl_engine.log
SSLLogLevel error

##
## SSL Virtual Host Context
##

<VirtualHost 192.168.100.1:80>

DocumentRoot /var/www
ServerAdmin karkoma@karkomaonline.com
ServerName www.karkomaonline.com
ErrorLog /var/log/httpd/error.log
CustomLog /var/log/httpd/access.log common

<Location /server-status>
SetHandler server-status
Order deny,allow
Deny from all
Allow from 192.168.100.
</Location>
…..

SSLEngine on
SSLProtocol SSLv3

SSLCipherSuite HIGH:MEDIUM

SSLCertificateFile /path/to/your/ssl/www.karkomaonline.com.crt
SSLCertificateKeyFile /path/to/your/ssl/www.karkomaonline.com.key
SSLCACertificateFile /path/to/your/ssl/karkomaCA/karkomaCA.crt
SSLVerifyClient require
SSLVerifyDepth 1
…..

</VirtualHost>

Generating your own certificate infrastructure

Depending on you needs you must pay a fee to Certificate Authority such as Thawte or VeriSign to get your site widely recognized as a secure site by an international CA. Or you can become a CA and create your own client and/or server certificates, sign and distribute them for your intranet for example or other needs.

Let’s start by creating our CA key and certificate:

openssl genrsa -rand /var/log/messages -out karkomaCA.key 1024
openssl req -new -x509 -key karkomaCA.key -out karkomaCA.crt

Okay, we have our private key and the corresponding certificate which will be used to sign the Certificate Signing Requests (CSR) requested by our clients. Note that in the above httpd.conf sample file SSLCACertificateFile directive points to this cert, this is the root certificate for the client certificates you’ll sign karkomaCA.crt. You could also add other CA’s root certificate if you plan to use them.

Now we are going to create a key and a Certificate Signing Request for the Apache webserver and sign it with our CA certificate; karkomaCA.

openssl genrsa -rand /var/log/messages -out www.karkomaonline.com.key 1024
openssl req -new -key www.karkomaonline.com.key -out www.karkomaonline.com.csr

openssl x509 -req -days 365 -in www.karkomaonline.com.csr -CA karkomaCA.crt -CAkey karkomaCA.key -CAcreateserial -out www.karkomaonline.com.crt

Finally create a CSR from your client and sign it with your CA cert:

openssl genrsa -rand /var/log/messages -out client.key 1024
openssl req -new -key client.key -out client.csr

openssl x509 -req -days 365 -in client.csr -CA karkomaCA.crt -CAkey karkomaCA.key -CAcreateserial -out client.crt

Now you are able to handle your sensitive information from your servers and clients in a more accurate form. You can implement this in your organization’s intranet as a means of handling private documents, test your internal developments based on SSL, etc. Please, see the References below to get an indepth knowledge of the SSL technology.

That’s all for now. Your comments are very welcome!

References

• The OpenSSL project homepage
• This is the Apache module that enables the webserver to communicate through Secure Sockets Layer
• RFC 2246 specifies Version 1.0 of the Transport Layer Security (TLS) protocol and here SSLv3 specification.
• RFC 2818 specifies HTTP over TLS