Sunday, September 1, 2019

Building Filebeat 8 on Rasbian

Filebeat now has prebuilt ARM binaries that can be installed directly from the repos. 

First I want to thank Michael Blouin and Michael Ross for their posts that largely guided me through figuring this out.
For this guide I will be using Raspbian Buster Lite 2019-07-10.  On a Raspberry Pi 4 4GB(this guide won't work on units with 1 GB of ram), you will need sudo access and please remember to change the password for 'pi' or better create a new user and remove 'pi'.
Installing Go 1.12.9, that latest build I could find for ARM
Building Elastic filebeat 8.0.0, the current version on github right now

Prep

We are going to need git.  It is not on Raspbian Lite out of the box.
sudo apt -y install git

Uninstall Go

We will want to remove any older versions.  Currently 1.11 is installed on Raspbian and doesn't support some features that are needed for filebeat 8.0.0.
sudo which go && sudo rm -rf /usr/bin/go* /usr/local/go


Install Go

At the time of this writing 1.12.9 was the newest version of Go I could find already built for ARM.  Why build it from source if we don't need to.
cd $HOME/
wget https://dl.google.com/go/go1.12.9.linux-armv6l.tar.gz
sudo tar -C /usr/local -xzf go1.12.9.linux-armv6l.tar.gz
export PATH=$PATH:/usr/local/go/bin
export GOPATH=$HOME/go
go version
go clean -cache

Download beats source

Lets download the latest beats source 8.0.0 at the time of this write up.
mkdir -p $HOME/go/src/github.com/elastic/
cd $HOME/go/src/github.com/elastic/
git clone https://github.com/elastic/beats


Build filebeat

cd beats/filebeat/
sudo su
export PATH=$PATH:/usr/local/go/bin
export GOPATH=/home/pi/go
make
./filebeat version
exit


Install filebeat

Nope, no nice .deb file here but it is easy to manually install.
sudo mkdir /usr/share/filebeat /usr/share/filebeat/bin /etc/filebeat /var/log/filebeat /var/lib/filebeat
sudo mv filebeat /usr/share/filebeat/bin
sudo mv module /usr/share/filebeat/
sudo mv modules.d/ /etc/filebeat/
sudo cp filebeat.yml /etc/filebeat/
sudo chmod 750 /var/log/filebeat
sudo chmod 750 /etc/filebeat/
sudo chown -R root:root /usr/share/filebeat/*

Create filebeat Service

Now that we have filebeat installed we will want to create a system service so we can manage it like any other service.
sudo nano /lib/systemd/system/filebeat.service
{paste}
[Unit]

Description=filebeat
Documentation=https://www.elastic.co/guide/en/beats/filebeat/current/index.html
Wants=userwork-online.target
After=network-online.target
[Service]
ExecStart=/usr/share/filebeat/bin/filebeat -c /etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs /var/log/filebeat
Restart=always
[Install]
WantedBy=multi-user.target

{end paste}
Save and close out of Nano
Ctrl+x
Y
enter

sudo systemctl enable filebeat.service
sudo service filebeat start
sudo service filebeat status

Now we have filebeat working!  Time to configure /etc/filebeat/filebeat.yml for your log files and destination host.


Clean up

If everything is working we can do a bit of cleanup.
cd $HOME
rm $HOME/go1.12.9.linux-armv6l.tar.gz
sudo rm -rf $HOME/go

2 comments:

  1. Hey James, Im glad I found your guides as I am trying to set up filebeat on my pi that has zeek already installed. I am running into an error. I am at the step of building filebeat, at the make command I am getting the error below:

    ../libbeat/logp/core.go:41:18: undefined: log.Writer
    make: *** [../libbeat/scripts/Makefile:125: filebeat] Error 2

    Would you have any idea on this error? Thank you.




    ReplyDelete
    Replies
    1. https://github.com/RaoulDuke-Esq/Beats-Pi

      Check this out it makes installing filebeat on a Pi as easy as apt-get install filebeat :P

      Delete

UniFi Network force DNS over TLS network wide

The post on doing this in OPNSense has been pretty popular so let's do this with UniFi!  You will need a UniFi gateway and the Network a...