Saturday, March 15, 2008

Setup a Mail Server in FreeBSD with Postfix and Dovecot

This tutorial show you how to configure a FreeBSD Mail Server using Postfix as a MTA (Mail Transfer Agent) and Dovecot as IMAP and POP3 Server.


Step 1. Compile and install Postfix from ports
--------------------------------------------------------------

cd /usr/ports/mail/postfix
make install


Step 2. Configure Postfix
----------------------------------

Create or modify file: /usr/local/etc/postfix/main.cf, to have the following content:

# -------------------- main.cf -----------------------------------------
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
mail_owner = postfix
unknown_local_recipient_reject_code = 550
mynetworks_style = host

debug_peer_level = 2

debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5

sendmail_path = /usr/local/sbin/sendmail
mailq_path = /usr/local/bin/mailq
setgid_group = maildrop
html_directory = no
manpage_directory = /usr/local/man
sample_directory = /usr/local/etc/postfix
readme_directory = no

mydomain = example.com
myhostname = firewall.$mydomain
myorigin = $myhostname
mydestination = $myhostname, localhost,$mydomain

mynetworks = 10.0.0.5/32
mynetworks_style = subnet
# --------- end of file main.cf --------------------


Add the following line to /etc/rc.conf
postfix_enable="YES"


Edit /usr/local/etc/rc.d/postfix and change variable postfix_enable to "YES"

Notes:
Note1
--------
Don't forget to setup in main.cf the variable: mydestination = $myhostname, localhost,$mydomain
If you forget to do that, you will get rely denied error, and emails that comes to your domain name will be rejected.
If you have many domains you can use transport_maps variable:

mydestination = $mydomain, $myhostname, $transport_maps, localhost

Transport maps can be also used from within mysql (for large mail servers with postfix compiled and configured with virtual users and MySQL support. If you are interested there is a tutorial on that topic on our website.

Note2
--------
Make sure you either add your IP in main.cf with your mynetworks variable or setup smtp server with authentication and allow your username to use SMTP with authentication.

mynetworks = 10.0.0.2/32


Step 3. Start postfix
---------------------------
Run command:

/usr/local/etc/rc.d/postfix start

If you have a problem when starting postfix or trying to send email, postfix is not working and you get a message in /var/log/messages like this:

postfix/smtpd[6877]: fatal: open database /etc/aliases.db: No such file or directory

create an empty file /etc/aliases and then recreate aliases DB with command:

postmap /etc/aliases

At this point you should have a full working mail transport agent (MTA).
The next step is to configure a POP3/IMAP client so we (mail clients) could get emails from mail server.


Step 4. Install Dovecot POP3/IMAP Server
-------------------------------------------------------

cd /usr/ports/mail/dovecot
make install

This will start fetch/compilation/installation process of Dovecot from Ports.

Edit /etc/rc.conf and add:
dovecot_enable="YES"
to start dovecot at boot.

Edit /usr/local/etc/rc.d/dovecot, and change dovecot_enable variable to "YES".

Create a file /usr/local/etc/dovecot.conf with the following content:

# ----------------- /usr/local/etc/dovecot.conf ---------------------
protocols = imap pop3
disable_plaintext_auth = no
ssl_disable = yes

mail_location = mbox:~/mail/:INBOX=/var/mail/%u
mail_privileged_group = mail
verbose_proctitle = yes
first_valid_gid = 0

protocol imap {
  imap_client_workarounds = delay-newmail outlook-idle netscape-eoh tb-extra-mailbox-sep
}

protocol pop3 {
  pop3_uidl_format = %08Xu%08Xv
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
}

protocol lda {
  postmaster_address = postmaster@example.com This email address is being protected from spam bots, you need Javascript enabled to view it
  sendmail_path = /usr/sbin/sendmail
}

auth default {
  mechanisms = plain

  passdb pam {
  }

  userdb passwd {
  }

  user = root
}
dict {
  #quota = mysql:/usr/local/etc/dovecot-dict-quota.conf
}

plugin {
}
# ----------------- end /usr/local/etc/dovecot.conf ----------------

Save file and start dovecot:

/usr/local/etc/rc.d/dovecot start

You should now have a full working mail server. If you've configured an email client and you are not able to connect to your mail server check /var/log/messages and /var/log/maillog files for errors.

After configuring your mail server you should be able to:
- get emails from your mail server using an email client, configured to use an account from mail server using POP3 or IMAP protocols (for this tutorial it will work with FreeBSD system accounts, if you want to configure an email server to work with virtual users see other tutorials on our website).
- send emails using SMTP server from the mail server you've configured;
- the mail server will be able to receive emails sent from other mails server/Internet.
 

Setup a Spam Filter with SpamAssasin with Postfix

If you receive a lot of spams here is a quick solution to filter spams using SpamAssasin and Postfix. Also spamd will be used (which is include within SpamAssasin). There are other implementations that we will cover in other tutorials.

Step 1. Install Postfix
-----------------------------
We will install Postfix from Ports:

cd /usr/ports/mail/postfix
make install

If you want to use Postfix with MySQL, you must check MySQL before compilation, also you may want to check Dovecot, to use Dovecot SASL authentication method. We recommend you dovecot over cyrus-imap.

After installation, configure postfix by editing: /usr/local/etc/postfix/master.cf and /usr/local/etc/postfix/main.cf

# -------------------- main.cf -----------------------------------------
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
mail_owner = postfix
unknown_local_recipient_reject_code = 550
mynetworks_style = host

debug_peer_level = 2

debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5

sendmail_path = /usr/local/sbin/sendmail
mailq_path = /usr/local/bin/mailq
setgid_group = maildrop
html_directory = no
manpage_directory = /usr/local/man
sample_directory = /usr/local/etc/postfix
readme_directory = no
#smtpd_recipient_restrictions = check_relay_domains, permit

mydomain = example.com
myhostname = firewall.$mydomain
myorigin = $myhostname
mydestination = $myhostname, localhost,$mydomain

mynetworks = 10.0.0.5/32
mynetworks_style = subnet
# --------- end of file main.cf --------------------

Also add the following line to /etc/rc.conf
postfix_enable="YES"

Notes:
Note1
--------
Don't forget to setup in main.cf the variable: mydestination = $myhostname, localhost,$mydomain
If you forget to do that, you will get rely denied error, and emails that comes to your domain name will be rejected.
If you have many domains you can use transport_maps variable:

mydestination = $mydomain, $myhostname, $transport_maps, localhost

Transport maps can be also used from within mysql (for large mail servers with postfix compiled and configured with virtual users and MySQL support. If you are interested there is a tutorial on that topic on our website.

Note2
--------
Make sure you either add your IP in main.cf with your mynetworks variable or setup smtp server with authentication and allow your username to use SMTP with authentication.

mynetworks = 10.0.0.2/32


Step 2. Install SpamAssasin
---------------------------------------

cd /usr/ports/mail/p5-Mail-SpamAssassin
make install


Step 3. Configure SpamAssasin
--------------------------------------------

After compilation and installation of SpamAssasin (which is written in Perl) from ports, as described in Step 2, create a config file for spamassasin:

touch /usr/local/etc/mail/spamassasin/local.cf

with the following content:
# ---------- local.cf --------------
rewrite_header Subject *****SPAM*****

# trusted_networks 10.0.0.
# lock_method flock
# use_bayes 1
# bayes_auto_learn 1

required_score 5.0
report_safe     1

whitelist_from dan@example.com This email address is being protected from spam bots, you need Javascript enabled to view it
# --------- end of local.cf

If you want to catch more spams, but also you will have more false positive, lower required_score. Higher score will catch less spams.


Step 4. Start spamd
----------------------------
Edit /usr/local/etc/rc.d/sa-spamd and change spamd_enable to "YES".
Then, start spamd:

/usr/local/etc/rc.d/sa-spamd start


Step 5. Edit master.cf and add support for spamassasin
-------------------------------------------------------------------
Edit /usr/local/etc/postfix/master.cf, replace line:

smtp      inet  n       -       n       -       -       smtpd
with:
smtp      inet  n       -       -       -       -       smtpd
        -o content_filter=spamassassin

And then add the following lines to master.cf.

spamassassin unix -     n       n       -       -       pipe
        user=spamd argv=/usr/local/bin/spamc -f -e
        /usr/sbin/sendmail -oi -f ${sender} ${recipient}


Step 6. Restart postfix and test the setup
---------------------------------------------------------

/usr/local/etc/rc.d/postfix restart
 

Setup a Syslog Server

Step 1. Edit /etc/syslog.conf
-------------------------------------
Edit /etc/syslog.conf and add the following lines:

# ------------------------ /etc/syslog.conf -------------------------------
#    Consult the syslog.conf(5) manpage.

+10.0.0.1
*.*                        /var/log/server1.log

+10.0.0.2
*.*                        /var/log/server2.log

+10.0.0.3
*.*                        /var/log/server3.log

*.err;kern.warning;auth.notice;mail.crit        /dev/console
*.notice;authpriv.none;kern.debug;lpr.info;mail.crit;news.err    /var/log/messages
security.*                    /var/log/security
auth.info;authpriv.info                /var/log/auth.log
mail.info                    /var/log/maillog
lpr.info                    /var/log/lpd-errs
ftp.info                    /var/log/xferlog
cron.*                        /var/log/cron
*.=debug                    /var/log/debug.log
*.emerg                        *

!startslip
*.*                        /var/log/slip.log

!dhcpd
*.*                        /var/log/dhcp.log
 # --------------------- eof -------------------------------------------------



Step 2. Add syslogd_flags="" in /etc/rc.conf
----------------------------------------------------------
Edit /etc/rc.conf and add the following line:
syslogd_flags=""

This will make syslog to listen external logging messages. By default this variable is defined in /etc/defaults/rc.conf with value "-s" to ignore external messages.
After editing /etc/rc.conf file, either reboot your machine or run: /etc/netstart.


Step 3. Restart syslog daemon
------------------------------------------
Make sure that server1.log, server2.log and server3.log files exists and then restart your syslog daemon (if those files does not exist you must create them):

/etc/rc.d/syslogd restart

Syslog uses UDP port 514, make sure this is not blocked from your firewall.

To send logs to this LOG server you must configure the other servers/devices from your network to send logs to this server. You can send logs from Linux, UNIX, BSD machines or from managed devices from your network (for example managed switches or routers). It will work with any syslog client.


Troubleshooting the log server
--------------------------------------------
To troubleshoot if your LOG server receives log message run tcpdump from root shell on port 514. Asuming your network card is em0:

tcpdump -tlni em0 port 514

If you do not receive messages after you run tcpdump then the problem is from your syslog client from device/server you want to log messages from. Check settings on that machine. Also, in next section of this tutorial is a small howto about configuring syslog clients to send messages to your syslog server.

If you do receive messages but syslog does not log them to your defined file, check if you have put in /etc/rc.conf syslog_enable="" and you've restarted the machine or reload network settings.

You can also test logging with logger command (available also on Linux).


Configuring syslog clients to send messages/logs to your syslog server
----------------------------------------------------------------------------------------------------

1. Sending LOGS messages from a FreeBSD server or desktop machine

Put the following line in /etc/syslog.conf and then restart your syslog daemon:

*.*         @10.0.0.10
(then restart your  syslog daemon on client with /etc/rc.d/syslogd restart)

If you want to send only kernel messages, put instead:

kern.*         @10.0.0.10


In this example 10.0.0.10 is the IP of your LOG Server.


2. Sending LOGS messages from a Linux server or desktop machine

Put the following line in /etc/syslog.conf (or /etc/syslogd.conf file, depending on your Linux distribution) and then restart your syslog daemon:
*.*         @10.0.0.10

(then restart your  syslog daemon on client with /etc/init.d/syslogd restart, this command is for Debian Linux, on other distribution it will be different)


3. Sending LOGS messages from a managed switch
You must access your managed switch using your telnet / ssh or web based administation console/interface and enable syslog client to send logs to other machine and configure it to send logs to your LOGS Server. Be carefull to allow that machine to be accepted by your managed switch otherwise it will not work. After setup, save your settings to NVRAM and reboot your device. Test functionality with tcpdump from your LOG Server. You must receive log messages.

To log messages from a Cisco routers or switch use (IOS):
config term
logging 10.0.0.10
logging trap errors
service timestamp log datetime
logging on

or (CatOS):

set logging server 10.0.0.10
set logging server severity 1
set logging timestamp enable
set logging server enable



4. Logging Windows Server/Desktop

By default syslog is not supported by windows. Still there might be third party applications that allows you to send syslog messages to a log server.

For example this project: http://www.syslogserver.com/download.html
 

Coming to FreeBSD from Linux

This tutorials is a short (and I hope useful) introduction on FreeBSD for people who come from Linux world.

Many things are similar in both FreeBSD and Linux operating systems. Some things might look weird, first time for somebody who come from Linux and do not have knowledge of other UNIX-like operating systems.

For a detalied handbook on FreeBSD visit: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/

To keep this intro short, here are some things you must know:

General things:
---------------------
After installing FreeBSD and activating SSH, you must create a user account in group wheel. By default FreeBSD installation of SSHd do not allow you to remotely connect via SSH on root, you must SSH as a user and then su to root. Also, FreeBSD will not allow any users to su to root. Your created user must be in group wheel to be allowed to su to root.

Networking:
-----------------

In Linux you have eth0 for first network card, eth1 for second and so on. In FreeBSD things are a little different, you will have different names for network interfaces, name given by manufacturer of that network cards. For example rl0 is first network Realtek 8139 interface, rl1 second and so on. Intel 100 mbps network cards are fxp (fxp0, fxp1 and so on), Intel Gigabit network cards are em0, em1 and so on. Here is a link with drivers (and naming) for ethernet cards: http://www.freebsd.org/releases/6.2R/hardware-i386.html#ETHERNET 


Hard Drives:
-----------------

In Linux you have hda for first IDE hard drive, hdb for second and so on, also you have sda for first scsi/sata, sdb for second scsi/sata hard drive. In FreeBSD you have ad0 for first IDE drive, ad1 for second IDE drive, da0 for first scsi drive, da1 for second and so on. For SATA drives you have ad, usualy numbering starts from ad4 (yes, it is ad like IDE) if you have IDE controller activated in bios, well it depends on motherboard model, and configuration of bios.

In FreeBSD CD/DVD drives are acd (acd0 first drive, acd1 second drive and so on).


Drive Partitions
----------------------

In Linux you have hda1 for first partition, hda2 for second and so on.

In FreeBSD things are a little bit different. You have slices and partitions. Partitions are created within a slice.
To understand better I'll give you an example:

An IDE (or SATA) drive of 200 GB size you have: ad0 (drive name)
We asume you only have one slice on total size of hard drive. That slice is ad0s1
Ok, then we asume on that slice we have 5 partitions: 1. root (mounted as / ), 2. swap, 3. tmp (mounted as /tmp ), 4. var (mounted as /var) and 5. usr, (mounted as /usr).

Then we will have
ad0s1a      root partition        mounted in /
ad0s1b      swap partition
ad0s1c      raw partition
ad0s1d     tmp partition         mounted in /tmp
ad0s1e     var partition         mounted in /var
ad0s1f      usr partition        mouned in /usr


Configuring network
-----------------------------

To configure nameservers: both Linux and FreeBSD use the same /etc/resolv.conf file, with the same content: "nameserver IP" (for example, IP is 10.0.0.1 and is the dns server that will be used). Multiple name servers can be used.

To configure network cards in Linux, it depends on the Linux distribution. For example in Debian you must edit /etc/network/interfaces file, and setup there either static IP or DHCP, then restart network service by using /etc/init.d/networking restart command.

Following our example in Debian Linux you must have the following lines in /etc/network/interfaces:

# --------- /etc/network/interfaces ------
# Loopback device:
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
  address 10.0.0.2
  broadcast 10.0.0.255
  netmask 255.255.255.0
  gateway 10.0.0.1
# -------- eof ----------------------------------

In FreeBSD network interface can be setup from /etc/rc.conf file which is main FreeBSD configuration files. Here can also be defined other things like what services to start on boot, parameters for that service, screen saver, language for terminal, and other FreeBSD options.

Following our example in FreeBSD you must have the following lines in /etc/rc.conf

# ------------ /etc/rc.conf ---------------
defaultrouter="10.0.0.1
ifconfig_fxp0="inet 10.0.0.1  netmask 255.255.255.0"
...
...
# -------- eof -----------------------------

To manualy add a default route in Linux use: route add default gw 10.0.0.1
To manualy add a default route in FreeBSD use: route add default 10.0.0.1
(we asume 10.0.0.1 is IP of the gateway).


Configuration files
--------------------------
In Linux most of configuration files are located in /etc, and in other subdirectories of /etc directory.

In FreeBSD only system configuration files are located in /etc. All application (userland) configuration files are located in /usr/local/etc. This aproach keeps the system cleaner.

Services are started in FreeBSD from /usr/local/etc/rc.d. In that directory are copied scripts that will start userland applications. FreeBSD base services must be started from /etc/rc.d directory.


Editors:
---------
To edit a file in FreeBSD use either edit or vi (installed by default). You can also install nano or joe from packages or ports. Also mcedit from Midnight Commander is available if you install mc package / port.

Firewalls
------------
Linux: iptables / netfilter.

FreeBSD uses ipfw (native FreeBSD firewall), ipfilter (native on FreeBSD but it seems not to be used anymore, the interesting thing is that was ported to other unices, like Solaris, HP/UX, OpenBSD,NetBSD) and PF (OpenBSD's PF, packet filter, ported to FreeBSD. Usualy either ipfw or PF is used, both have similar features.

More info can be found here:

IPFW: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/firewalls-ipfw.html
IPFW man page: man ipfw

PF: http://www.openbsd.org/faq/pf/
PF man page: man pf.conf


Userland Applications, ports and packages
-----------------------------------------------------------
For this tutorial we will still use our example, comparing FreeBSD with Debian Linux.

In Debian you have dpkg, apt-get and aptitude for package management, packages have extension .deb.

In FreeBSD you have packages and ports. Packages are precompiled FreeBSD applications that can be installed using pkg_install command.  FreeBSD's packages extension is .bz2, but those packages are not regular bz2 archives, it also contain an index recognized by pkg_install application, and script files that will be run at installation time.  Port system on the other hand make possible fetch and compilation of applications before installation.

Examples of package management in Debian
-----------------------------------------------------
apt-get install ftpd         # will install ftpd application under Debian Linux
apt-get remove ftpd      # will remove ftpd application from Debian Linux system
dpkg -i ftpd                      # install ftpd application under Debian using dpkg
dpkg -r ftpd                     # remove ftpd application under Debian using dpkg
dpkg -L ftpd                    # find files installed by ftpd package under Debian
dpkg -l | grep ftpd         # find ftpd package from Debian installed packages
                                         # (find if ftpd package is installed or not)
dpkg -s ftpd                   # get info about ftpd package
dpkg -S /bin/netstat     # find to which package belongs netstat binary
aptitude install ftpd     # will install ftpd package
aptitude remove ftpd  # will remove ftpd package
aptitude seach ftpd     # will search for ftpd package
 
Examples of package management in FreeBSD
--------------------------------------------------------
pkg_add -r apache                   # install apache package
pkg_delete apache-2.2.4_2     # delete apache package
pkg_info                                     # list installed packages
pkg_info | grep apache           # find if apache package is installed
pkg_info -L apache-2.2.4_2     # list files installed by apache package (you must use the exact
                                                     # name of the package, if your version differs use previous
                                                     # command to find exact name of package.
pkg_info -r apache-2.2.4_2      # will show on which packages depend apache package

For more info consult man pages for pkg_add, pkg_delete, pkg_info, pkg_create, pkg_version.

Examples of managing ports in FreeBSD
------------------------------------------------
If you do not want do install an application from packages, for example you need to recompile a package with a specific option, you have the possibility to install it from ports. For example we want to install apache22:

whereis apache22               # we will get apache22: /usr/ports/www/apache22
cd /usr/ports/www/apache22
make install                         
# this will fetch source files  for apache and will compile and then
                                                # install apache. If other ports will be needed by apache,
                                                # those ports will be installed too.
make                                       # only fetch and compile but not install port
make clean                             # remove fetched/compiled port files
make deinstall                   
# deinstall port, but not remove from ports
make reinstall                         # deinstall and reinstall port

Note1: After you've installed a port, it wil be seen as a package too, using pkg_info command.
Note2: In order to be able to install ports, you must choose to install ports collection when you first install FreeBSD.


Configuring and starting a FreeBSD application after it was installed from packages or ports
------------------------------------------------------------------------------------------------------------------------------
As I've told you before, all configuration files for userland applications under FreeBSD are keeped in /usr/local/etc
(with few minor exceptions)

Configuring an application under  FreeBSD consist of following steps:
- we asume you've installed application using pkg_install, or make install from ports
- find and edit config file (or files) that comes with that application
- edit /usr/local/etc/rc.d/servicename file and  change  servicename_enable  from  "NO" to  "YES"
- edit /etc/rc.conf file and add servicename_enable="YES" if you want that service to start on boot
- run service: /usr/local/etc/rc.d/servicename start


FreeBSD Kernel
---------------------
By default at installation a GENERIC kernel is used by FreeBSD. If you want to add some features to FreeBSD kernel (support for different options that are not in GENERIC) you have two possibilities:
- Load a kernel module for that option/feature
- Recompile the kernel

Load a kernel module:
kldload module.ko              # load a kernel module
kldstat                                # list loaded modules
kldunload module.ko          # unload a module

Modules can be found in /boot/kernel/.

Compiling FreeBSD kernel:
(we asume you have installed i386 version of FreeBSD)
a) Copy and edit Kernel config file
cp /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/SERVER

edit /usr/src/sys/i386/conf/SERVER
Add your option to SERVER file, then save file.

b) Compile Kernel
cd /usr/src
make -j4 buildkernel KERNCONF=SERVER
make installkernel KERNCONF=SERVER

 

Other useful things in FreeBSD
-------------------------------------------
Logs
are generated using syslog facility. To configure syslog service, edit /etc/syslog.conf file. To restart syslog daemon, run /etc/rc.d/syslogd restart.

Cron service is managed by crond daemon. To configure you can use /etc/crontab file (not recommended by some syadmins because if you cvsup and recompile your source tree you might loose your change, but to use instead a crontab file in /var/cron/tabs. To restart cron service /etc/rc.d/cron restart.

Sysctl can be use to tune/change many FreeBSD sysctl variables.
sysctl -a                                        # will show all sysctl variables
sysctl -w variable=value            # will change a sysctl variable
To make sysctl to be setup at boot time put the variable=value line on a single line in /etc/sysctl.conf. Some sysctl values can only be changed before /etc scripts are processed, in that case you must put the sysctl variable=value in /etc/loader.conf file.
 

Setup Load Balanced PPPoE Servers with OSPF

This tutorial is about how to setup multiple PPPoE Servers accessible from LAN as a load balanced system, all servers connected to the same router.

Content
-----------

1. Introduction: purpose of the project
2. The simplest setup for a PPPoE Server
3. Network Topology for Load Balanced PPPoE Servers
4. Technologies we used
5. Setting up the Router
6. Setup the PPPoE servers
7. Setup Radius and MySQL
8. Setup Router and PPPoE servers to work with OSPF
9. Firewalls and Traffic Shaping
10. Testing


1. Introduction: purpose of the project
----------------------------------------------------

On medium or large networks an easy way to offer Internet access to private users or to companies is to use PPPoE servers. This allows customers to authenticate using a username and a password, then to login to PPPoE server and have an Internet connection.

PPPoE stands for Point to Point over Ethernet. This technology can be used to authenticate users connected to your own ISP using ADSL, Cable Modem infrastructure or even on Ethernet. So it is possible to offer Internet connection to users from local Ethernet networks, using PPPoE.


2. The simplest setup for a PPPoE server
--------------------------------------------------------
To learn how to setup a PPPoE server go to Setup a PPPoE Server with MPD tutorial . You will find there the basics of setting up a PPPoE server.


 

3. Network Topology for Load Balanced PPPoE Servers
--------------------------------------------------------------------------

The previous example will work for a small network, still it does not have any firewall or traffic shaping. It is useful to understand how to setup a FreeBSD PPPoE server. From this section a real case example will be presented, with complete configuration files.

Next picture shows a network topology for a small or medium size ISP that will use PPPoE servers for their customers:
 

ISP Network Topology PPPoE Servers FreeBSD, Load  Balancing

This is an example, in real world setup, topology might be different in complexity.


4. Technologies we used
----------------------------------
- Operating System: FreeBSD 6.x (on Router and on PPPoE Servers)
- Firewall and traffic shaper on Router: OpenBSD's PF (packet filter)
- Firewall and traffic shaper on PPPoE Servers: IPFW
- MPD for PPPoE Server - mpd-4.3
- OSPF routing protocol (with quagga) - quagga-0.99.8_2
- radius server - freeradius-1.1.4_1
- all users from LAN have asigned public IPs, routed to Routers's public IP. Also we have a small public subnet that we used for Router and PPPoE servers

The purpose of this setup is to asign public IPs to lan computers and to load balance PPPoE servers. If you do not have public IPs for every user, you do not need to use OSPF routing protocol, just asign a private subnet for every pppoe server and it will work.

PPPoE server respond to requests from clients on LAN interfaces in a round robin way, and asign IP to lan clients. If you reboot one server all connected users to that server will be disconnected and then they will reconnect to other server from pool.

 

PPPoE server pools, load balancing freebsd




If you've read previous version you noticed network topology. For our tutorial we simplified previous picture so our network topology is presented in the next picture:


Asuming that our config informations are:

Router
---------

Public IP of router (fxp0): 80.10.10.2 gateway 80.10.0.1 netmask 255.255.255.0 (this is the line that comes from our ISP)
Public IP 2 of router (fxp1): 80.80.0.1 netmask 255.255.255.224 (this is connected to PPPoE servers)
Hostname: router.example.com

PPPoE Server 1
---------------------
Public IP (fxp0) 80.80.0.2 netmask 255.255.255.224 (this is interface conected to Router and other PPPoE servers)
No IP for LAN interface (fxp1)

PPPoE Server 2
---------------------
Public IP (fxp0) 80.80.0.3 netmask 255.255.255.224 (this is interface conected to Router and other PPPoE servers)
No IP for LAN interface (fxp1)

PPPoE Server 3
---------------------
Public IP (fxp0) 80.80.0.4 netmask 255.255.255.224 (this is interface conected to Router and other PPPoE servers)
No IP for LAN interface (fxp1)


5. Setting up the Router
--------------------------------

Config files will be presented next for Router.

# ----------- rc.conf file ----------------------
defaultrouter="80.10.10.1"
gateway_enable="YES"
hostname="router.example.com"
ifconfig_fxp0="inet 80.10.10.2  netmask 255.255.255.0"
ifconfig_fxp1="inet 80.80.0.1  netmask 255.255.255.224"
sshd_enable="YES"
usbd_enable="YES"
pf_enable="YES"
pf_rules="/etc/pf.conf"
# -----------------end rc.conf------------------
 


# ------------- pf.conf file --------------------
ext_if="fxp0"
int_if="fxp1"
pass quick all
# ----------------end pf.conf------------------

6. Setup the PPPoE servers
-------------------------------------

We will show you the config files for PPPoE 1. You will have to config the rest of your PPPoE servers in similar way, changing only IP of PPPoE server, according to diagram (or of course to your particular setup).

# ----------- rc.conf file PPPoE 1 ----------------------
defaultrouter="80.80.0.1"
gateway_enable="YES"
hostname="pppoe.example.com"
ifconfig_fxp0="inet 80.80.0.2  netmask 255.255.255.0"
ifconfig_fxp1="inet 10.0.0.1  netmask 255.255.255.0"
sshd_enable="YES"
usbd_enable="YES"
firewall_enable="YES"
firewall_script="/etc/rc.firewall"
quagga_enable="YES"
quagga_flags="-d"
quagga_daemons="zebra ospfd"
watchquagga_enable="YES"

# -----------------end rc.conf------------------
 

# -------------- mpd.conf file -----------------

default:
 log auth iface

 load s0
 load s1
 load s2
 load s3
 

s0:
 new -i ng0 s0 ppplink0
 load generic

s1:
 new -i ng1 s1 ppplink1
 load generic

s2:
 new -i ng2 s2 ppplink2
 load generic

s3:
 new -i ng3 s3 ppplink3
 load generic
 

generic:
    set iface enable proxy-arp
    set pppoe iface fxp1
    set iface idle 0
    set iface mtu 1462
    set iface enable tcpmssfix
    set link accmap 0
    set link enable pap
    set link enable chap
    set link accept chap-msv2
    set link max-redial -1
    set link keep-alive 10 80
    set link no acfcomp
    set link no protocomp
    set ipcp no vjcomp
    set ipcp dns 80.20.0.1 80.20.0.2
    set bundle enable noretry
    set bundle enable multilink
    set auth max-logins 1
    set pptp disable windowing
    set pptp enable always-ack
    set iface up-script /usr/local/etc/mpd4/addclient.sh
    set iface down-script /usr/local/etc/mpd4/removeclient.sh
    set auth enable radius-auth
    set auth enable radius-acct
    set radius server 80.80.0.5 secret_password
    set radius timeout 7
    set radius retries 5
 

# ---------------- end mpd.conf file ----------

# --------------- mpd.links file ----------------

ppplink0:
 set link type pppoe
 set pppoe iface fxp1
 set pppoe service "*"
 set pppoe enable incoming

ppplink1:
 set link type pppoe
 set pppoe iface fxp1
 set pppoe service "*"
 set pppoe enable incoming

ppplink2:
 set link type pppoe
 set pppoe iface fxp1
 set pppoe service "*"
 set pppoe enable incoming

ppplink3:
 set link type pppoe
 set pppoe iface fxp1
 set pppoe service "*"
 set pppoe enable incoming

# --------------end mpd.links file -------------

 7. Setup Radius and MySQL
------------------------------------
 

# ------------------- /usr/local/etc/raddb/clients.conf ----------

client 80.80.0.2 {
        secret          = secret_password
        shortname       = pppoe1
        nastype     = other     # localhost isn't usually a NAS...
}

client 80.80.0.3 {
        secret          = secret_password
        shortname       = pppoe2
        nastype     = other     # localhost isn't usually a NAS...
}


client 80.80.0.4 {
        secret          = secret_password
        shortname       = pppoe3
        nastype     = other     # localhost isn't usually a NAS...
}
 

 # ------------------- end clients.conf ---------------------------

Edit in /usr/local/etc/raddb/sql.conf the following:

        # Connect info
        server = "localhost"
        login = "raduser"
        password = "password"

        # Database table configuration
        radius_db = "radius"
 

 Use your database information instead.

 Modify radius.conf file. The configuration file is quite large so you can view it here

Create a new database for radius with clients information with the following structure:

radacct -table used for logging purposes
radcheck - table that keeps usernames and passwords
radgroupcheck - configurations for groups
radgroupreply - configurations for groups
radpostauth - table used for logging purposes
radreply - table that keeps the ips of the users
usergroup - shows the group each user belongs to
 

create database radius;

use radius; 

-- ------------- Table structure for table `radacct` -------------------

CREATE TABLE `radacct` (
  `RadAcctId` bigint(21) NOT NULL auto_increment,
  `AcctSessionId` varchar(32) NOT NULL default '',
  `AcctUniqueId` varchar(32) NOT NULL default '',
  `UserName` varchar(64) NOT NULL default '',
  `Realm` varchar(64) default '',
  `NASIPAddress` varchar(15) NOT NULL default '',
  `NASPortId` int(12) default NULL,
  `NASPortType` varchar(32) default NULL,
  `AcctStartTime` datetime NOT NULL default '0000-00-00 00:00:00',
  `AcctStopTime` datetime NOT NULL default '0000-00-00 00:00:00',
  `AcctSessionTime` int(12) default NULL,
  `AcctAuthentic` varchar(32) default NULL,
  `ConnectInfo_start` varchar(32) default NULL,
  `ConnectInfo_stop` varchar(32) default NULL,
  `AcctInputOctets` bigint(12) default NULL,
  `AcctOutputOctets` bigint(12) default NULL,
  `CalledStationId` varchar(50) NOT NULL default '',
  `CallingStationId` varchar(50) NOT NULL default '',
  `AcctTerminateCause` varchar(32) NOT NULL default '',
  `ServiceType` varchar(32) default NULL,
  `FramedProtocol` varchar(32) default NULL,
  `FramedIPAddress` varchar(15) NOT NULL default '',
  `AcctStartDelay` int(12) default NULL,
  `AcctStopDelay` int(12) default NULL,
  PRIMARY KEY  (`RadAcctId`),
  KEY `UserName` (`UserName`),
  KEY `FramedIPAddress` (`FramedIPAddress`),
  KEY `AcctSessionId` (`AcctSessionId`),
  KEY `AcctUniqueId` (`AcctUniqueId`),
  KEY `AcctStartTime` (`AcctStartTime`),
  KEY `AcctStopTime` (`AcctStopTime`),
  KEY `NASIPAddress` (`NASIPAddress`)
) ENGINE=InnoDB  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1214869 ;

 

-- ----------------- Table structure for table `radcheck` --------------------------

CREATE TABLE `radcheck` (
  `id` int(11) unsigned NOT NULL auto_increment,
  `UserName` varchar(64) NOT NULL default '',
  `Attribute` varchar(32) NOT NULL default '',
  `op` char(2) NOT NULL default '==',
  `Value` varchar(253) NOT NULL default '',
  PRIMARY KEY  (`id`),
  KEY `UserName` (`UserName`(32))
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=5811 ;

-- ---------------  Table structure for table `radgroupcheck` ------------------

CREATE TABLE `radgroupcheck` (
  `id` int(11) unsigned NOT NULL auto_increment,
  `GroupName` varchar(64) NOT NULL default '',
  `Attribute` varchar(32) NOT NULL default '',
  `op` char(2) NOT NULL default '==',
  `Value` varchar(253) NOT NULL default '',
  PRIMARY KEY  (`id`),
  KEY `GroupName` (`GroupName`(32))
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=4 ;

-- ----------------- Table structure for table `radgroupreply` ---------------

CREATE TABLE `radgroupreply` (
  `id` int(11) unsigned NOT NULL auto_increment,
  `GroupName` varchar(64) NOT NULL default '',
  `Attribute` varchar(32) NOT NULL default '',
  `op` char(2) NOT NULL default '=',
  `Value` varchar(253) NOT NULL default '',
  `prio` int(10) unsigned NOT NULL default '0',
  PRIMARY KEY  (`id`),
  KEY `GroupName` (`GroupName`(32))
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=9 ;

-- ------------------ Table structure for table `radpostauth` ------------

CREATE TABLE `radpostauth` (
  `id` int(11) NOT NULL auto_increment,
  `user` varchar(64) NOT NULL default '',
  `pass` varchar(64) NOT NULL default '',
  `reply` varchar(32) NOT NULL default '',
  `date` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB  DEFAULT CHARSET=latin1 AUTO_INCREMENT=1203292 ;

-- ------------  Table structure for table `radreply` ----------------

CREATE TABLE `radreply` (
  `id` int(11) unsigned NOT NULL auto_increment,
  `UserName` varchar(64) NOT NULL default '',
  `Attribute` varchar(32) NOT NULL default '',
  `op` char(2) NOT NULL default '=',
  `Value` varchar(253) NOT NULL default '',
  PRIMARY KEY  (`id`),
  KEY `UserName` (`UserName`(32))
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=6283 ;

-- ----------- Table structure for table `usergroup` -------------

CREATE TABLE `usergroup` (
  `id` int(11) unsigned NOT NULL auto_increment,
  `UserName` varchar(64) NOT NULL default '',
  `GroupName` varchar(64) NOT NULL default '',
  `priority` int(11) NOT NULL default '1',
  PRIMARY KEY  (`id`),
  KEY `UserName` (`UserName`(32))
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 AUTO_INCREMENT=7505 ;
 

Create one group for all users, we'll call it dynamic:

INSERT INTO `radgroupcheck` (`id`, `GroupName`, `Attribute`, `op`, `Value`) VALUES
(2, 'dynamic', 'Simultaneous-Use', ':=', '1'),
(1, 'dynamic', 'Auth-Type', '==', 'Local');
 

INSERT INTO `radgroupreply` (`id`, `GroupName`, `Attribute`, `op`, `Value`, `prio`) VALUES
(1, 'dynamic', 'Framed-Compression', ':=', 'Van-Jacobsen-TCP-IP', 0),
(2, 'dynamic', 'Framed-Protocol', ':=', 'PPP', 0),
(3, 'dynamic', 'Service-Type', ':=', 'Framed-User', 0),
(4, 'dynamic', 'Framed-MTU', ':=', '1500', 0),
(5, 'dynamic', 'X-Ascend-Assign-IP-Pool', ':=', '0', 0),
(6, 'dynamic', 'X-Ascend-Maximum-Time', ':=', '7200', 0),
(8, 'dynamic', 'Idle-Timeout', ':=', '1800', 0);
 

 

Add a users to the database. To add a user you need to enter the following data into the database:
 

insert into radcheck values ("", "customer1", "Password","==","password");
insert into radreply values ("", "customer1", "Framed-IP-Address",":=","80.0.0.20");
insert into usergroup values('', 'customer1', 'dynamic', 1);

 

8. Setup Router and PPPoE servers to work with OSPF
-------------------------------------------------------------------------
Add following to rc.conf

quagga_enable="YES"
quagga_flags="-d"
quagga_daemons="zebra ospfd"
 

Edit the file /usr/local/etc/rc.d/quagga and replace the line:

 : ${quagga_enable="NO"}

to

: ${quagga_enable="YES"} 

And then start quagga.

/usr/local/etc/quagga start 

Here are the configuration files needed to setup OSPF. You need to make similar configuration files for each PPPOE server and for router.
 

# ------------------------- zebra.conf ---------------------- 

hostname PPPOE1-zebra
password zebra_password
enable password zebra_enable_password
!log file /var/log/zebra.log
interface fxp0
!
interface fxp1
!
interface lo0
!
ip forwarding
ipv6 forwarding
!
line vty
 

# ------------------------- end zebra.conf ----------------------

# ------------------------- ospfd.conf ----------------------

hostname PPPOE1-ospf
password ospfd_password
enable password ospfd_enable_password
log file /var/log/ospf.log
!
interface fxp0
!
interface fxp1
!
interface lo0
!
router ospf
 ospf router-id 80.80.0.2
 redistribute connected
 passive-interface ng0
 passive-interface ng1
 passive-interface ng2
 passive-interface ng3
 network 80.80.0.0/27 area 0.0.0.0
neighbor 80.80.0.3 
!
line vty
!
 

# ------------------------- end ospfd.conf ----------------------


9. Firewalls and Traffic Shaping
------------------------------------------

# ------------------------- rc.firewall ----------------------

#!/bin/sh
cmd="/sbin/ipfw -q"
lif="fxp1"
lif_ng="ng*"
pif="fxp0"
 

# we have 2 types of subscriptions which will have different bandwidth setups
ab500="80.80.0.0/25"
ab350="80.80.0.128/25"
net="80.80.0.128/24"

# bandwidth extern
down500_extern="8000kbits/s"
up500_extern="2000kbits/s"
down350_extern="4000kbits/s"
up350_extern="2000kbits/s"

# metropolitan bandwidth
down500_metro="20000kbits/s"
up500_metro="10000kbits/s"
down350_metro="10000kbits/s"
up350_metro="10000kbits/s"


queue="64"
gred="0.02/6/18/0.1"

#flush rules
$cmd flush
$cmd pipe flush

# table for allow fixed ip users
#$cmd table 1 flush

#table for warned users (clients that did not paid in time)
$cmd table 2 flush

#table for metropolitan networks
#$cmd table 3 flush

# load metro networks
/etc/rc.tables

# pipe 1 --> internet pipe (35M)
# pipe 10 --> metropolitan pipe (90M)
# ------------------- extern ---------------------
# pipe 2 --> pipe download ab500 - mask 0x0000ffff 4096K
# pipe 3 --> pipe download ab350 - mask 0x0000ffff 1024K
# pipe 4 --> pipe upload ab500 - mask 0x0000ffff 512K
# pipe 5 --> pipe upload ab350 - mask 0x0000ffff 256K
# queue 1 --> down prioritized traffic - mask 0x0000ffff weight 15
# queue 2 --> down not prioritized traffic - mask 0x0000ffff weight 10

# ------------------- metropolitan ---------------------
# pipe 12 --> pipe download ab500 - mask 0x0000ffff 10240K
# pipe 13 --> pipe download ab350 - mask 0x0000ffff 10240K
# pipe 14 --> pipe upload ab500 - mask 0x0000ffff 4096K
# pipe 15 --> pipe upload ab350 - mask 0x0000ffff 4096K

# queue 10 --> down prioritized traffic - mask 0x0000ffff weight 15
# queue 11 --> down not prioritized traffic - mask 0x0000ffff weight 10

# --------------------- EXTERN---------------------
$cmd pipe 1 config bw 45000Kbit/s

#queues download
$cmd queue 1 config weight 15 pipe 1 queue 100 mask dst-ip 0xffffffff
$cmd queue 2 config weight 10 pipe 1 queue 50  mask dst-ip 0xffffffff

# pipe for download ab500
$cmd pipe 2 config bw $down500_extern mask dst-ip 0xffffffff queue $queue gred $gred
#upload ab500
$cmd pipe 4 config bw $up500_extern mask src-ip 0xffffffff queue $queue gred $gred

# pipe for download ab350
$cmd pipe 3 config bw $down350_extern mask dst-ip 0xffffffff queue $queue gred $gred
#upload ab350
$cmd pipe 5 config bw $up350_extern mask src-ip 0xffffffff queue $queue gred $gred

# --------------------- METROPOLITAN---------------------
$cmd pipe 10 config bw 120000Kbit/s

#queues download
$cmd queue 10 config weight 15 pipe 10 queue 100 mask dst-ip 0xffffffff
$cmd queue 11 config weight 10 pipe 10 queue 50  mask dst-ip 0xffffffff

# pipe for download ab500
$cmd pipe 12 config bw $down500_metro mask dst-ip 0xffffffff queue $queue gred $gred
#upload ab500
$cmd pipe 14 config bw $up500_metro mask src-ip 0xffffffff queue $queue gred $gred

# pipe for download ab350
$cmd pipe 13 config bw $down350_metro mask dst-ip 0xffffffff queue $queue gred $gred
#upload ab350
$cmd pipe 15 config bw $up350_metro mask src-ip 0xffffffff queue $queue gred $gred


#permit localhost
$cmd add 10 allow ip from any to any via lo0
$cmd add 20 deny ip from any to 127.0.0.1

$cmd add 7 allow ip from any to me
$cmd add 8 allow ip from me to any

#  allow multicast for ospf
$cmd add 130 allow ip from any to 224.0.0.0/29
 

# antispoof
$cmd add 45 deny all from 172.16.0.0/12 to any via $lif
$cmd add 45 deny all from 192.168.0.0/16 to any via $lif
#$cmd add 50 deny ip from any to any not verrevpath in

#redirect warned clients
$cmd add 60 forward 127.0.0.1,8010 ip from "table(2)" to any dst-port 80

# fixed ips allow
$cmd add 100 allow ip from any to "table(1)" out xmit $lif
$cmd add 110 allow ip from "table(1)" to any in recv $lif
$cmd add 120 skipto 60000 ip from "table(1)" to any in recv $lif


# antivirus
$cmd add 3600 deny ip from any 135-139,445,1080,2283,2745,2745,3127-3198,5554,8866,10080,65506 to any in
# block mirc
$cmd add 3700 deny ip from any to any dst-port 6666, 6667, 6668, 6669, 7000, 9000

# deny all except ng* on lan
$cmd add 3800 deny ip from any to any in recv $lif
 

# ---------------- EXTERN TRAFFIC --------------------

# download clients ab500
$cmd add 4100 pipe 2 ip from not "table(3)" 80,443,53 to $ab500 in recv $pif
$cmd add 4200 queue 1 ip from not "table(3)" 80,443,53 to $ab500 in recv $pif
$cmd add 4300 skipto 60000 ip from not "table(3)" 80,443,53 to $ab500 in recv $pif

# default traffic
$cmd add 4400 pipe 2 ip from not "table(3)" to $ab500 in recv $pif
$cmd add 4500 queue 2 ip from not "table(3)" to $ab500 in recv $pif

# upload clients ab500
$cmd add 5200 pipe 4 ip from $ab500 to not "table(3)" out xmit $pif

#download clients ab350
$cmd add 6000 pipe 3 ip from not "table(3)" 80,443,53 to $ab350 in recv $pif
$cmd add 6100 queue 1 ip from not "table(3)" 80,443,53 to $ab350 in recv $pif
$cmd add 6200 skipto 60000 ip from not "table(3)" 80,443,53 to $ab350 in recv $pif
# default traffic
$cmd add 6300 pipe 3 ip from not "table(3)" to $ab350 in recv $pif
$cmd add 6400 queue 2 ip from not "table(3)" to $ab350 in recv $pif

#upload clients ab350
$cmd add 6500 pipe 5 ip from $ab350 to not "table(3)" out xmit $pif

# ---------------- METROPOLITAN TRAFFIC --------------------

# download clients ab500
$cmd add 7100 pipe 12 ip from "table(3)" 80,443,53 to $ab500 in recv $pif
$cmd add 7200 queue 10 ip from "table(3)" 80,443,53 to $ab500 in recv $pif
$cmd add 7300 skipto 60000 ip from "table(3)" 80,443,53 to $ab500 in recv $pif

# default traffic
$cmd add 7400 pipe 12 ip from "table(3)" to $ab500 in recv $pif
$cmd add 7500 queue 11 ip from "table(3)" to $ab500 in recv $pif

# upload clients ab500
$cmd add 7600 pipe 14 ip from $ab500 to "table(3)" out xmit $pif

#download clients ab350
$cmd add 8000 pipe 13 ip from "table(3)" 80,443,53 to $ab350 in recv $pif
$cmd add 8100 queue 10 ip from "table(3)" 80,443,53 to $ab350 in recv $pif
$cmd add 8200 skipto 60000 ip from "table(3)" 80,443,53 to $ab350 in recv $pif
# default traffic
$cmd add 8300 pipe 13 ip from "table(3)" to $ab350 in recv $pif
$cmd add 8400 queue 11 ip from "table(3)" to $ab350 in recv $pif

#upload clients ab350
$cmd add 8500 pipe 15 ip from $ab350 to "table(3)" out xmit $pif

# allow traffic on ng*
$cmd add 60000 allow ip from any to $net out xmit $lif_ng
$cmd add 60100 allow ip from $net to any in recv $lif_ng

$cmd add 60200 allow ip from any to $net in recv $pif
$cmd add 60300 allow ip from $net to any out xmit $pif

$cmd add 61000 deny ip from any to any
 

# ------------------------- end rc.firewall ----------------------

# ----------------------- rc.tables ---------------------- 

#!/bin/sh
cmd="/sbin/ipfw -q"

# fixed ips allowed
$cmd table 1 flush

#table for metropolitan networks
$cmd table 3 flush

#se definesc table cu ip-uri permise
$cmd table 1 add 80.80.0.10
$cmd table 1 add 80.80.0.11

#metro
$cmd table 3 add 80.10.0.1/24
$cmd table 3 add 80.15.0.1/24

# ----------------------- end rc.tables ----------------------

10. Testing
---------------

Command to see connected users:

/usr/local/bin/mysql --password=mysql_password --exec='use radius; select username from radacct where AcctStopTime=0;'

 This setup uses mysql information to check if the user is still connected. If you want to use radzap script to remove a connected user from radius server, the default script will not update the mysql database, so radius will think it is still connected. To change the script to update mysql database you can add the following line at the end of the script:

 `/usr/local/bin/mysql --password=mysql_password --exec='use radius; update radacct set AcctStopTime=AcctStartTime where AcctStopTime=0 and UserName like "'$user'";'`

 

 This script executes radzap for all connected users. If you want it to run at boot time place it it /usr/local/etc/rc.d folder.

# --------------------- radzap-all.sh script -----------------------

#!/bin/sh

for i in `radwho | grep -v "Login" | cut -d" " -f1`
do
        `/usr/local/bin/radzap -u $i localhost secret_password`
done

`/usr/local/bin/mysql --password=mysql_password --exec='use radius; update radacct set AcctStopTime=AcctStartTime where AcctStopTime=0;'`

 # --------------------- radzap-all.sh script -----------------------

FreeBSD Router with Traffic Shaping with PF and ALTQ HFSC

Here is a tutorial about building a FreeBSD router with traffic shaping using OpenBSD's PF and ALTQ HFSC discipline.

Step 1. Compile Kernel with support for PF and ALTQ
-------------------------------------------------------------------------


cd /usr/src/sys/i386/conf/
cp GENERIC ROUTER

edit ROUTER file and add the following lines at the end of file:

# ------------------ add the following lines to ROUTER file ------------------
# pf support
device mem
device pf
device pflog
device pfsync

# altq support
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_PRIQ

# other optimizations
options HZ=1000
options DEVICE_POLLING
# ---------------------------------- eof ----------------------------------------------


Next, compile kernel with configurations from ROUTER file

cd /usr/src
make -j4 buildkernel KERNCONF=ROUTER
make installkernel KERNCONF=ROUTER

Reboot the machine and you have support in kernel for PF and ALTQ


Step 2 Create pf.conf file for your firewall and traffic shaper
---------------------------------------------------------------------------------

Rename your default /etc/pf.conf file and create a new config file. In our example we asume your network cards are fxp0 for WAN and fxp1 for LAN. also your LAN subnet is 192.168.0.0/24, and we setup LAN IP of router with value 192.168.0.1. Our LAN being on a private subnet (we only have one public IP) we will use NAT from PF.

Shaping rules are for two PCs on LAN. Both have asigned a maximum of 5Mb bandwidth, with a guaranteed bandwidth of 1Mb

Next is presented pf.conf file:

# --------------------- pf.conf ---------------------
ext_if="fxp0"
int_if="fxp1"
pc1="192.168.0.2"
pc2="192.168.0.3"

altq on $ext_if hfsc bandwidth 10Mb queue {def_up,pc1_up, pc2_up}
altq on $int_if hfsc bandwidth 10Mb queue {def_down,pc1_down, pc2_down}


queue pc1_up bandwidth 5Mb hfsc(realtime 1Mb linkshare 50% upperlimit 5Mb)
queue pc2_down bandwidth 5Mb hfsc(realtime 1Mb linkshare 50% upperlimit 5Mb)

queue def_up bandwidth 128Kb hfsc(realtime 128Kb linkshare 10% upperlimit 256Kb default)
queue def_down bandwidth 128Kb hfsc(realtime 128Kb linkshare 10% upperlimit 256Kb default)

nat on $ext_if from $int_if:network to any -> ($ext_if)

# ------ Pass rules, Shaping for PC1
pass in quick on $ext_if from any to $pc1
pass out quick on $int_if from any to $pc1 queue pc1_down

pass in quick on $int_if from $pc1 to any
pass out quick on $ext_if from $pc1 to any queue pc1_up

# ------ Pass rules, Shaping for PC2
pass in quick on $ext_if from any to $pc2
pass out quick on $int_if from any to $pc2 queue pc2_down

pass in quick on $int_if from $pc2 to any
pass out quick on $ext_if from $pc2 to any queue pc2_up

block all
# ----------------------- end pf.conf file ---------------------------


Step 3. Edit your /etc/rc.conf file and enable pf at startup to load config from /etc/pf.conf file
----------------------------------------------------------------------------------------------------------------------------

Your rc.conf file should look like this:

# -------------- rc.conf -----------------
hostname="router.example.com"
gateway_enable="yes"
defaultrouter="80.80.0.1"

ifconfig_fxp0="inet 80.80.0.2 netmask 255.255.255.224"
ifconfig_fxp1="inet 192.168.0.1 netmask 255.255.255.0"

sshd_enable="yes"

pf_enable="YES"
pf_rules="/etc/pf.conf"
# ---------------- end rc.conf ---------



Tips to debug PF rules:
--------------------------------


pfctl -vvsr       (see PF loaded rules)
pfctl -vvsq       (see PF queues in realtime)
pfctl -f /etc/pf.conf      (load pf.conf file)
pfctl -F state               (flush states)
 

FreeBSD Router with Traffic Shaping with PF and ALTQ CBQ

Here is a tutorial about building a FreeBSD router with traffic shaping using OpenBSD's PF and ALTQ CBQ discipline.

Step 1. Compile Kernel with support for PF and ALTQ
-------------------------------------------------------------------------


cd /usr/src/sys/i386/conf/
cp GENERIC ROUTER

edit ROUTER file and add the following lines at the end of file:

# ------------------ add the following lines to ROUTER file ------------------
# pf support
device mem
device pf
device pflog
device pfsync

# altq support
options ALTQ
options ALTQ_CBQ
options ALTQ_RED
options ALTQ_RIO
options ALTQ_HFSC
options ALTQ_PRIQ

# other optimizations
options HZ=1000
options DEVICE_POLLING
# ---------------------------------- eof ----------------------------------------------


Next, compile kernel with configurations from ROUTER file

cd /usr/src
make -j4 buildkernel KERNCONF=ROUTER
make installkernel KERNCONF=ROUTER

Reboot the machine and you have support in kernel for PF and ALTQ


Step 2 Create pf.conf file for your firewall and traffic shaper
---------------------------------------------------------------------------------

Rename your default /etc/pf.conf file and create a new config file. In our example we asume your network cards are fxp0 for WAN and fxp1 for LAN. also your LAN subnet is 192.168.0.0/24, and we setup LAN IP of router with value 192.168.0.1. Our LAN being on a private subnet (we only have one public IP) we will use NAT from PF.

Shaping is for 2 PCs on LAN. We've used a bandwidth of 10Mb/s, we've asigned 45% of bandwidth for every PC, and 10% for default queue.

Next is presented pf.conf file:

# --------------------- pf.conf ---------------------
ext_if="fxp0"
int_if="fxp1"
pc1="192.168.0.2"
pc2="192.168.0.3"

altq on $ext_if cbq bandwidth 10Mb queue {def_up, pc1_up, pc2_up}
altq on $int_if cbq bandwidth 10Mb queue {def_down, pc1_down, pc2_down}

queue def_up bandwidth 10% cbq(default)
queue def_down bandwidth 10% cbq(default)

queue pc1_up bandwidth 45% priority 6 cbq(red)
queue pc1_down bandwidth 45% priority 6 cbq(red)
queue pc2_up bandwidth 45% priority 6 cbq(red)
queue pc2_down bandwidth 45% priority 6 cbq(red)

nat on $ext_if from $int_if:network to any -> ($ext_if)

pass in quick on $ext_if from any to $pc1
pass out quick on $int_if from any to $pc1 queue pc1_down

pass in quick on $int_if from $pc1 to any
pass out quick on $ext_if from $pc1 to any queue pc1_up

pass in quick on $ext_if from any to $pc2
pass out quick on $int_if from any to $pc2 queue pc2_down

pass in quick on $int_if from $pc2 to any
pass out quick on $ext_if from $pc2 to any queue pc2_up

block all
# ----------------------- end pf.conf file ---------------------------


Step 3. Edit your /etc/rc.conf file and enable pf at startup to load config from /etc/pf.conf file
----------------------------------------------------------------------------------------------------------------------------

Your rc.conf file should look like this:

# -------------- rc.conf -----------------
hostname="router.example.com"
gateway_enable="yes"
defaultrouter="80.80.0.1"

ifconfig_fxp0="inet 80.80.0.2 netmask 255.255.255.224"
ifconfig_fxp1="inet 192.168.0.1 netmask 255.255.255.0"

sshd_enable="yes"

pf_enable="YES"
pf_rules="/etc/pf.conf"
# ---------------- end rc.conf ---------



Tips to debug PF rules:
--------------------------------


pfctl -vvsr       (see PF loaded rules)
pfctl -vvsq       (see PF queues in realtime)
pfctl -f /etc/pf.conf      (load pf.conf file)
pfctl -F state               (flush states)
 

Firewall for Web Server with PF

Step 1. Create a file /etc/pf.conf (just rename the old one) and put the following content in it:
------------------------------------------------------------------------------------------------------------------------------

wan_if="em0"
priv_nets="{10.0.0.0.8, 172.16.0.0/12, 192.168.0.0/16, 224.0.0.0/4, 240.0.0.0/5, 127.0.0.0/8, 0.0.0.0}"
tcp_ports="{ 20, 21, 22, 25, 53, 80, 110, 143 }"
udp_ports="{ 53 }"
icmp_types="8"

table <rfc1918> const { 192.168.0.0/16, 10.0.0.0/8, 172.16.0.0/12 }
table <blocklist> persist file "/etc/blocklist.txt"

set timeout { interval 10, frag 30 }
set timeout { tcp.first 120, tcp.opening 30, tcp.established 86400 }
set timeout { tcp.closing 900, tcp.finwait 45, tcp.closed 90 }
set timeout { udp.first 60, udp.single 30, udp.multiple 60 }
set timeout { icmp.first 20, icmp.error 10 }
set timeout { other.first 60, other.single 30, other.multiple 60 }
set timeout { adaptive.start 0, adaptive.end 0 }
set limit { states 15000, frags 30000 }
set optimization normal
set block-policy drop
set fingerprints "/etc/pf.os"
set loginterface $wan_if
set require-order yes

scrub in on $wan_if all no-df fragment reassemble min-ttl 15 max-mss 1460
scrub in on $wan_if all no-df
scrub    on $wan_if  all reassemble tcp

block in quick on $ext_if from $priv_nets

pass in all
pass out all

pass quick on lo0 all

block in log all
block out log all

block in quick on $wan_if from <blocklist> to any
block in quick on $wan_if from <rfc1918> to any

pass in on $wan_if proto tcp from any to $wan_if port $tcp_ports flags S/SA synproxy state
pass in on $wan_if proto udp from any to $wan_if port $udp_ports keep state
pass in on $wan_if inet proto icmp all icmp-type $icmp_types keep state
pass in inet proto icmp icmp-type 8 code 0 keep state



pass in quick on $wan_if proto tcp from any to any port > 49151  keep state

pass out on $wan_if proto tcp all modulate state flags S/SA
pass out on $wan_if proto { udp, icmp } all keep state


Step 2: Add the following line to /etc/rc.conf:
------------------------------------------------------------
pf_enable="YES"


Step 3: Eenable PF firewall and load rules
----------------------------------------------------------
pfctl -e
pfctl -f /etc/pf.conf

(you will need to have /etc/blocklist.txt file created)
 

Firewall for Web Server with IPFW

Here are presented IPFW firewall rules for a Web Server.

Step 1: Create a file: /etc/rc.firewall then put the following content in that file
--------------------------------------------------------------------------------------------------------

(replace em0 with your network interface)

#!/bin/sh
cmd="/sbin/ipfw -q"
wan_if="em0"

$cmd flush
$cmd pipe flush

$cmd allow ip from any to any via lo0
$cmd add check-state

$cmd add reset tcp from any to any established

# allow HTTP traffic
$cmd add allow tcp from any to me 80 setup in keep-state

# allow DNS
$cmd add allow udp from any to me 53 in keep-state
$cmd add allow tcp from any to me 53 setup in keep-state

# allow SMTP
$cmd add allow tcp from any to me 25 setup in keep-state
$cmd add allow tcp from any to me 22 setup in keep-state

# allow FTP
$cmd add allow tcp from any to me 21 setup in keep-state
$cmd add allow tcp from any to me 20 setup in keep-state
$cmd add allow tcp from me 20 to any setup out keep-state

#allow POP3
$cmd add allow tcp from any to me 110 setup in keep-state
#allow IMAP
$cmd add allow tcp from any to me 143 setup in keep-state

# allow ping
$cmd add allow icmp from any to me icmptypes 8 in keep-state

# allow traffic to server
$cmd add allow tcp from me to any setup out keep-state
$cmd add allow ip from me to any out keep-state
$cmd add allow tcp from any to me setup in keep-state
$cmd add allow ip from any to me in keep-state

# deny everything else
$cmd add deny tcp from any to any setup
$cmd add deny ip from any to any


Step 2: Add the following lines in rc.conf
-------------------------------------------------------

firewall_enable="YES"
firewall_script="/etc/rc.firewall"


Step3: Run firewall script
-----------------------------------

/etc/rc.firewall
 

Configuring a DNS Server

Step 1. Create named.conf file with the following content:

Note: options "directory", ", "pid-file", "dump-file", "statistics-file" might have other values if you configure bind server on Linux. The following values are for FreeBSD.

Note: do not forget to put ";" after every IP, incuding last IP, and to enclose rules between { }.

 

options {
        directory       "/etc/namedb";
        pid-file        "/var/run/named/pid";
        dump-file       "/var/dump/named_dump.db";
        statistics-file "/var/stats/named.stats";

        forwarders      { 213.157.176.3; 213.157.176.1; };
        allow-recursion { 10.0.0.1/16; 127.0.0.1; };
        allow-transfer  { 213.157.176.3; 213.157.176.1; 192.162.16.0/24; };
        listen-on       { 127.0.0.1; 86.X.Y.Z; };
};

zone "." {
        type hint;
        file "named.root";
};

zone "0.0.127.IN-ADDR.ARPA" {
        type master;
        file "master/localhost.rev";
};


zone "Z.Y.X.86.in-addr.arpa" {
        type master;
        file "master/Z.Y.X.86.in-addr.arpa";
};

zone "domeniu.ro" in {
        type master;
        file "/etc/namedb/domeniu.ro";
};

Few explanations regarding following variables:

        forwarders      { 213.157.176.3; 213.157.176.1; };
        allow-recursion { 10.0.0.1/16; 127.0.0.1; };
        allow-transfer  { 213.157.176.3; 213.157.176.1; 192.162.16.0/24; };
        listen-on       { 127.0.0.1; 86.X.Y.Z; };

forwarders - here you place your ISP DNS Servers (or other DNS servers from root, that accept your IPs). This is also useful for DNS cache.
 

allow-recursion - allow only to IP placed here (or to subnets) to query the DNS server. You must place here all IPs or subnets that will use the DNS server.

allow-transfer - allow only to these servers to transfer zones from current DNS server (there can be DNS slave servers for example).

listen-on - the IP on which will run the DNS server.
 


Step 2. Create the file for the domain "example.com", (we asume example.com is the domain you want to setup) with the following content:

$TTL 3600        ; 1 ora
example.com.    IN      SOA      ns1.example.com. admin.example.com. (
                                2006051501      ; Serial
                                10800           ; Refresh
                                3600            ; Retry
                                604800          ; Expire
                                86400           ; Minimum TTL
                        )

; DNS Servers
                IN      NS      ns1.example.com.
                IN      NS      ns2.example.com.

; MX Records
                IN      MX 10   mx.example.com.
                IN      MX 20   mail.example.com.

                IN      A       86.X.Y.Z
; Machines
localhost       IN      A       127.0.0.1
ns1             IN      A       86.X.Y.Z
ns2             IN      A       86.X.Y.Z
mx              IN      A       86.X.Y.Z
mail            IN      A       86.X.Y.Z
; Aliases
www             IN      CNAME   @

Note: be careful not to omit "." when defining zone, after every host name. If you omit ".", bind will add after machine name the origin of zone (in our case example.com). So "." at the end of hostname in zone means the
exact name of hostname.


Step 3. Add in /etc/resolv.conf the following line:

nameserver 127.0.0.1


Step 4. Test the DNS server

After you've configured bind (named.conf) and you've created zone file you will start bind service to test if it works. After you've stared bind (/etc/rc.d/named -forcestart) you must have answer when pinging the domain name from localhost. Try to ping every hostname defined as A records in your zone file.

Note: NS and MX records from zone must all have defined A records in order to properly work. If NS and MX records do not have A records defined with the same name it won't answer to ping either from localhost or from outside and it won't work.

Test example (from localhost):
#ping ns1.example.com

If after configuration hosts defined in DNS server zone answers to ping then from localhost everything works well. To test from outside you must wait for DNS to propagate to the Internet. This will take up to 24 hours.

Then you must the proper answer to queries on zone records (NS, MX, subdomains).

If DNS server does not answers when you ping on A records from localhost:
- check if name server is started (ps awux | grep named)
- check if name server is listen on port 53 (netstat -an | grep 53)
- you can start named in foreground with "named -f" to check error messages or you can activate logs for named service)

As a note you should also have open port 953 which is used by rndc service to reload named server.

Test DNS server with dig:
 

dig -x @ domeniu.com any

If you do not want to see all records but only MX or NS replace "any" with NS or MX. If digs returns your records defined in your zone then you've succesfuly setup your DNS server, and you must wait for DNS records to propagate to the Internet.

Test DNS Server with nslookup:
 

#nslookup
>set q=any
>example.com
^D

Example:

webserver# nslookup
> set q=any
> example.com
Server:         127.0.0.1
Address:        127.0.0.1#53

example.com
        origin = ns1.example.com
        mail addr = webmaster.example.com
        serial = 2007061061
        refresh = 21600
        retry = 3600
        expire = 604800
        minimum = 86400
example.com    nameserver = ns1.example.com.
example.com    mail exchanger = 10 mail.example.com.
Name:   example.com
Address: 86.X.Y.Z

Query the DNS Server by using local DNS server:
 

> lserver example.com
Default server: example.com
Address: 86.X.Y.Z#53

Example DNS server query for MX records with nslookup:
 

# nslookup -type=mx example.com
Server:         127.0.0.1
Address:        127.0.0.1#53

example.com    mail exchanger = 10 mail.example.com.

Query the DNS server in verbose mode, useful for debug:

> set debug
> example.com
Server:         127.0.0.1
Address:        127.0.0.1#53

------------
    QUESTIONS:
        example.com, type = A, class = IN
    ANSWERS:
    ->  example.com
        internet address = 86.X.Y.Z
    AUTHORITY RECORDS:
    ->  example.com
        nameserver = ns1.example.com.
    ADDITIONAL RECORDS:
    ->  ns1.example.com
        internet address = 86.X.Y.Z
------------
Name:   example.com
Address: 86.X.Y.Z

Query of the DNS server in more verbose mode (debug 2):

> set d2
> example.com

Howto configure a Slave DNS server:

The Slave DNS server usualy is setup for redundancy. It will share the load with Master DNS server and will answer to DNS request if the Master DNS server is not accesible. Usualy is not recommended to use two Master DNS servers (it is possible). You can use multiple Slave DNS servers. A Slave DNS server can transfer DNS zones to other Slave DNS server (of course if it is configured to do that).

How it works: the Master DNS server read DNS records from file and then sends those records to the Slave DNS server. The zone file from Slave DNS server is a copy of the zone file from Master DNS server.

Example for Master and Slave DNS servers:

zone "example.com" in {
        type master;
        file "/etc/namedb/example.com";
zone "example.com" in {
        type slave;
        file "/etc/namedb/slave.example.com";
 masters { 86.X.Y.Z; };