Polymorphing for Linux Fedora Documentation

Polymorphing for Linux Overview

This section details how to install Polyverse’s polymorphic packages from an internet-connected machine. Polymorphing for Linux becomes the primary repository on the targeted system. This allows all supported packages to be retrieved from the Polyverse scrambled binary repository, and any custom, private, or unsupported packages to be retrievable from their original repositories. Every package downloaded is unique to the specific customer, and each used package is replaced every twenty-four hours.

Make sure your operating system is up-to-date

dnf upgrade -y

Install Polymorphing for Linux

Note: Replace the demo auth key with your own, user-specific auth key.

curl -s https://repo.polyverse.io/cli | sh -s install <Your auth key here>

Reinstall all packages

dnf clean all
dnf upgrade -y
dnf repository-packages polyverse-os move-to -y --exclude=kernel*
dnf repository-packages polyverse-updates move-to -y --exclude=kernel*

Please reboot after re-installation, unless you're running in a container.

The configuration and installation is complete at this point.

Overview

This section provides a simple approach to downloading Polymorphing for Linux packages from an internet-connected host, and then installing those packages on a non-internet-connected host.

At a high-level, the steps are:

• On an internet-connected host, install Polymorphing for Linux, download the .rpm packages to a folder, and create an index file.
• Copy the index and packages to a folder accessible by the non-internet-connected host.
• Install packages directly from the folder.

All the tools and techniques to provide this offline installation capability are standard approaches to how this can be done for any other RPM-based Linux repo. They can be adjusted for use in Docker containers, VMs, or bare metal.

Install Polymorphing for Linux and Download Packages on Internet Connected Host

Note: Replace the demo auth key with your own, user-specific auth key.

dnf install -y yum-plugin-ovl
dnf upgrade -y
dnf install -y dnf-plugins-core createrepo
# this is the demo key. please replace with the licensed authkey.
curl -s https://repo.polyverse.io/cli | sh -s install <Your auth key here>
dnf clean all
dnf makecache
dnf upgrade -y
dnf -y repository-packages polyverse-os move-to
dnf -y repository-packages polyverse-updates move-to
mkdir /opt/pv cd /opt/pv
# dnf wraps lines longer than 80 characters when piped, so we need this in order 
# to unwrap those lines.
dnf download $(dnf list installed | tr "\n" "#" | sed -e 's/# / /g' | tr "#" "\n" | grep polyverse | awk '{print $1}')
createrepo .

Install Polymorphing for Linux Packages on Non-Internet Connected Host

Note: Replace the URL in red with the location that you copied the .rpm files from /opt/pv to. Also, be sure you've disabled all other repositories. You can do this by going to /etc/yum.repos.d/, editing each .repo file, and adding enabled=0, or changing enabled=1 to enabled=0 for each repository.

dnf upgrade -y
vi /etc/yum.repos.d/polyverse.repo
# START CONTENTS OF polyverse.repo
[polyverse]
name=Polymorphic Fedora $releasever - $basearch
failovermethod=priority
baseurl=file:///opt/pv
enabled=1
gpgcheck=0
cost=1
# END CONTENTS OF polyverse.repo
dnf clean all
dnf -y repository-packages polyverse move-to
dnf list installed

Overview

This section details the steps for configuring and installing a local mirror of Polymorphing for Linux for Fedora. This is meant to synchronize packages for a given processor architecture. Setting up a variety of repositories for various Fedora versions and processor architectures isn't covered in this document.

Technical Overview

Polyverse uses four repositories for serving packages:

• polyverse-os
• polyverse-mirror-os
• polyverse-updates
• polyverse-mirror-updates

polyverse-os: This is the main repository for Polyverse's scrambled binaries. Packages for a release are stored here.
polyverse-mirror-os: This is a a mirror of the official CentOS repositories. This should always have more packages than polyverse-os, or the same amount.
polyverse-updates: This contains scrambled binaries for updates that occur between releases.
polyverse-mirror-updates: This is a mirror of the CentOS updates repository. Like polyverse-mirror-os, this should always have more packages than polyverse-updates, or the same amount.

Make sure your operating system is up-to-date

dnf upgrade -y

Install Polymorphing for Linux packages

Before setting up a mirror, we need to be sure the Polymorphing for Linux repositories are installed on the machine hosting the mirror. Follow these installation steps:
Note: Replace the demo auth key in red with your own, user-specific auth key.

curl -s https://repo.polyverse.io/cli | sh -s install <Your auth key here>
# reinstall all packages
dnf clean all
dnf upgrade -y
dnf repository-packages polyverse-os move-to -y --exclude=kernel*
dnf repository-packages polyverse-updates move-to -y --exclude=kernel*

Set up the local Yum Repository Server

Replicate the public mirror with reposync

Reposync is a tool that allows you to synchronize each local directory with your current yum repositories. We will be using this command to download all of the relevant packages from the currently configured yum repositories.

Install reposync

Reposync is installed with the yum-utils package. Make sure the yum-utils package is installed by running:

dnf install yum-utils createrepo

Install Apache

For this example, we want to serve the repository's packages over HTTP, or HTTPS, so we need to install Apache. If you want to serve the packages using a different protocol, skip the next three steps, but be aware that this will change the directory that you synchronize files to.

dnf install httpd

Start Apache

Because we will place the repository in /var/www/html which is the default directory for Red Hat based Apache installations; we will not need to change any of the apache configuration. When implementing this you may want to review the configuration to ensure that it meets your security requirements.

chkconfig httpd on
service httpd start

Configure the Firewall to Allow Connections to Apache

This guide uses http via port 80 to allow repository clients to connect and interact with the repository. The system firewall will block these connections by default, so the following command must be run to configure the system to allow connections on port 80.

iptables -I INPUT 4 -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT

Synchronize with the system’s currently configured repositories

We’re placing all of the synchronized packages into /var/www/html so that the directory can be presented by Apache to any consumers of the repository. If you are using another protocol, such as ftp or nfs, then this directory may differ.

reposync -l -d --newest-only --download-metadata -p /var/www/html/

Optionally, if you want to place specific repositories on their own servers, you can synchronize that repository's packages using the following commands:

reposync -l -d --repoid=<repo-id> --newest-only --download-metadata -p /var/www/html/

• -l enables yum plugin support
• -d enables deleting of local packages no longer present in repository
• --newest-only tell reposync to only pull the latest version of each package in the repos
• --download-metadata enables downloading all the non-default metadata
• -p specifies the path to download packages

Run this for each of the following repo IDs that you want to synchronize:

• polyverse-os
• polyverse-mirror-os
• polyverse-updates
• polyverse-mirror-updates

Each repo-id can be placed on the server of your choice.

Create Yum XML Metadata with createrepo

Yum leverages the createrepo tool to create the necessary XML metadata files for indicating what packages are available. Every time a repository is updated with new or removed packages, these XML metadata files will need to be updated with createrepo.

Note: Depending on how many packages your distro's repos have, createrepo may take a long time to finish.

createrepo /var/www/html/<repo-id>

You should do this for each repository.

Update the XML Metadata

In order to keep the yum repository updated, the reposync and createrepo utilities can be run as cron jobs. When updating the XML Metadata, it often saves time and reduces I/O operations by using the --update flag with the createrepo command.

createrepo --update /var/www/html/<repo-id>

This should be called for each repository.

Configure Yum Clients

Create the Repo Configuration File

Now that the Yum Repository server is setup, the clients will need to point to this new repository. This can be achieved by creating a repository configuration file.

vi /etc/yum.repos.d/polyverse.repo

Insert the following text into the /etc/yum.repos.d/polyverse.repo:

[polyverse-os]
name=polyverse-$releasever os
baseurl=http://ip_of_mirror_server/polyverse-os
gpgcheck=0
keepcache=0
enabled=1
priority=1

[polyverse-mirror-os]
name=polyverse-mirror-$releasever os
baseurl=http://ip_of_mirror_server/polyverse-mirror-os
gpgcheck=0
keepcache=0
enabled=1
priority=2

[polyverse-updates]
name=polyverse-$releasever updates
baseurl=http://ip_of_mirror_server/polyverse-updates
gpgcheck=0
keepcache=0
enabled=1
priority=1

[polyverse-mirror-updates]
name=polyverse-mirror-$releasever updates
baseurl=http://ip_of_mirror_server/polyverse-mirror-updates
gpgcheck=0
keepcache=0
enabled=1
priority=2

Notes about the configuration above:

• If you decide to store each repository on a different server, use the IP of that server in the appropriate repository's configuration block.
• The gpgcheck and gpgkey options are optional and can be enabled by setting gpgcheck=1.
• This example uses HTTP to serve the repositories for clarity's sake, but there's no reason not to use HTTPS if you can.

Disable the Default Repositories

After creating the new repository file, any default repositories should be disabled, since polyverse-mirror-os and polyverse-mirror-updates contain any official packages that haven't been scrambled yet. This can be done by editing all of the repository (.repo) files found in /etc/yum.repos.d/ EXCEPT for polyverse.repo, using the following command:

vi <default files>.repo

Find every repository in each .repo file, and add enabled=0, or change enabled=1 to enabled=0 to disable that repository.

Refresh Yum Cached Repository List

After disabling the default repositories, and enabling the new local repository, yum’s cached repository list will need to be updated. This can be done with the following command.

dnf makecache

Update and reinstall packages

At this point, you're ready to reinstall the packages being served from your new mirror.

dnf clean all
dnf upgrade -y
dnf repository-packages polyverse-os move-to -y --exclude=kernel*
dnf repository-packages polyverse-updates move-to -y --exclude=kernel*

Polymorphic (Scrambled) Linux Kernel

This section details how to install the Polyverse scrambled Kernel on a VMWare Fusion 10.x virtual machine setup.

Fedora 23, 24, and 25 Polymorphic Kernel

This process assumes a clean installation of Fedora.

Install the Debug Kernel

Since reinstalling the currently in-use kernel will cause a conflict, the debug kernel will need to be installed so that the scrambled kernel can replace the currently in-use kernel after a reboot.

1. Check the current kernel version:

   uname -a

2. Check the current kernel attributes, and record them somewhere for future comparison:

   file /boot/vmlinuz-$(uname -r)

3. Install the debug kernel:

   dnf install -y kernel-debug

4. Execute the following command from the VM terminal so that you can select the desired kernel for booting from the grub menu:

   reboot

5. Select the debug kernel version from the grub menu

Install Polyverse and the Scrambled Kernel

Note: Make sure that you have a valid Polyverse auth key to use. Replace the demo auth key with your individual auth key

1. Remove the original kernel packages:

   dnf remove -y kernel kernel-core kernel-modules

2. Install the Polyverse scrambled binary repository

   curl -s https://repo.polyverse.io/cli | sh -s install <Your auth key here> --add-component kernel

3. Install the kernel packages
   NOTE: Polyverse currently only supports kernels for the x86_64 architecture.

   dnf --disablerepo='*' --enablerepo='polyverse-updates-kernel' install -y kernel.x86_64

Run the Scrambled Kernel

1. Execute the following command from the VM terminal so that you can select the desired kernel for booting from the grub menu:

   reboot

2. Select the latest kernel from the grub menu

Verify that the Scrambled Kernel is Running

1. Check the current kernel version:

   uname -a

2. Check the current kernel attributes:

   file /boot/vmlinuz-$(uname -r)

3. Observe the difference between the build properties for the original kernel versus the Polverse scrambled kernel.

Uninstalling Polymorphing for Linux

Uninstalling Polyverse is a quick task. The high-level steps are as follows:

1. Remove any references to the Polyverse scrambled binary repository from the system repository configuration files or folders
2. Reinstall all packages so that they are downloaded from the remaining referenced repositories (which will not include Polyverse)

Remove the Polyverse Repository References

There is a script provided to automatically perform the removal of the Polyverse repository references. It can be executed using the following command:

curl -s https://repo.polyverse.io/cli | sh -s install --uninstall

Reinstall all packages

dnf upgrade -y
dnf reinstall -y \*
dnf list installed

The configuration and uninstallation is complete at this point.

Missing Packages

The package may not actually be available. Use this command to determine if the package is available for download from the repository.

dnf search <package_name>

Error: The operation would result in removing the booted kernel: kernel-core-<version>

When the console shows the following error, preventing a reinstallation of packages:

1. Remove python3-dnf-plugins-core.noarch using the following command

   dnf remove python3-dnf-plugins-core.noarch

2. Run the original command that resulted in the error to see if the issue was resolved

Public Issue Reference: https://bugzilla.redhat.com/show_bug.cgi?id=1348766

Error: package perl-Getopt-Long-2.49.1-1.fc25.noarch requires perl(Exporter), but none of the providers can be installed.

If this happens during the installation process, on the dnf repository-packages polyverse-os move-to -y --exclude=kernel* command, try removing the perl package, re-running the dnf repository-packages polyverse-os move-to -y --exclude=kernel*command, then re-installing the perl package again afterwards.