You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 37 Next »

Contents:

Related pages:

The root page KOP Knowledge Base could not be found in space kiuwan.


System Architecture Overview

Kiuwan On-Premises (KOP) is based on a client-server architecture with the following major server components:

  • WildFly application server 
  • MySQL database
  • Redis in-memory data store

KOP solution is deployed on a centralized server (all the components are installed in the same host) and accessed by users via HTTP website access or by IDE plugins (Eclipse, Visual Studio).

KOP also supports distributed architecture with high-availability and load-balancing. For these scenarios, please contact Kiuwan support.

Server Host Requirements

Installation requirements

Kiuwan On-Premises (KOP)  solution is deployed on a centralized server host (all the components are installed in the same host) and accessed by users via HTTP or by IDE plugins (Eclipse, Visual Studio).

KOP installation process consists on building a Docker image.

Server Host Requirements

  • Linux kernel version 3.10 or higher

  • CS Docker Engine version 1.10 or higher installed on your server 

  • Connectivity to SMTP Mail Server (i.e. remotely accessible or locally installed)

  • The first phase of installation process (building the Kiuwan image) requires Internet connectivity 


Installation must be executed by a 
unix user with privileges to execute docker server commands.

During the installation process, you will need to specify a KOP unix user and group :

  • KOP user will the owner of the KOP processes that will run within the docker container 
  • the docker persistent volume will be owned by KOP user

KOP User and Group is specified during the KOP Installation process as KIUWAN_USER and KIUWAN_GROUP properties

You can create KOP user and group by executing next commands (no special privileges are needed):

  • sudo groupadd ${KIUWAN_GROUP}
  • sudo useradd -g ${KIUWAN_GROUP} -s /bin/false -m -d /home/${KIUWAN_USER} ${KIUWAN_USER}


KOP Persistent Volume is created by the installation process and must specified as DIR_PERSISTENT_VOLUME property

Be sure to dispose of a minimum of 6 GB of free disk space :

  • 3 Gb for Docker image

  • 3 Gb for KOP Persistent Volume


KOP persistent volume contains KOP database and its size will increase as more Kiuwan analyses are executed.
  • Recommended size for KOP persistent volume is 50 Gb
  • Try to specify an independent disk/partition, so you can easily increase it later if needed

CPU and memory requirements

 Server CPU and memory requirements depend on several factors:

  • how many lines of code will need to be scanned
  • how many analyses will need to be executed in parallel 
  • how many users will access the web application

 

It's important to notice that we're assuming the typical use case where code analyses are performed locally through Kiuwan Local Analyzer, and therefore out of server machine (tipically at a user machine or within a build CI system).

Therefore, when we talk about analysis we mean 2nd phase of the analysis (i.e. indicator calculation that is executed within the host server)

 

Above factors heavily influence host server requirements, but minimum requirements are:

Operating SystemRAMCPUsDiskOther SW

Linux kernel version 3.10 or higher

16 GB

Intel i7 2,5 GHz with 4 cores

750 GBCS Docker Engine version 1.10 or higher

Note: CPU clock speed and disk speed will affect overal response time. 

 

Above configuration is recommended for a system load that requires continuous service for:

  • parallel execution of 2 analyses (any additional parallel analysis request will be enqueued, and it will be executed as soon as any of the running analyses finishes), and
  • 50 concurrent web-users

Additional system requirements can be met by following next recommendations:

  • In case you need a higher number of parallel analyses executions, you should add 1 CPU core and 512 Mb for any additional required parallel analysis
  • In case you need to improve response time to web users, conside to add 1Gb for every 50 additional web users.

 

Installation steps

Before installing KOP, make sure that you understand the System Architecture Overview, that your server host(s) complies with the Server Host Requirements, and that you have properly prepared the Environment as follows:

 

Before installing KOP, make sure your server host complies with the Server Host Requirements

 

Installation Steps:

 

Step 1. Unpackage KOP Installation Package

KOP Installation Package consists on a tarball gz file (docker-for-kiuwan_master.X.Y.tar.gz) containing all the kiuwan docker files.

  • Copy the distribution tarball to your host server’s installation directory ($KOP_INSTDIR)
  • Uncompress the distribution tarball  
    • tar xvzf docker-for-kiuwan_master.X.Y.tar.gz
  • This will create a $KOP_INSTDIR/docker-for-kiuwan_master.X.Y directory with all the kiuwan docker files

  • In $KOP_INSTDIR/docker-for-kiuwan_master.X.Y you will find a file named Dockerfile where you will configure the Kiuwan docker image.

 

KOP Licenses are distributed through a zip file that contains two license distribution zipfiles:

  • license.zip
  • configq1.zip

To install the licences:

  1. Copy both zipfiles to "license" directory of you KOP installation directory
    • cp license.zip $KOP_INSTDIR/docker-for-kiuwan_master.X.Y/license
    • cp configq1.zip $KOP_INSTDIR/docker-for-kiuwan_master.X.Y/license
  2. Continue configuring and building your KOP image

Please visit How to Install KOP Licenses is you need to reinstall (or upgrade) licenses after the docker container is built.

 

Step 2. Configure Kiuwan Docker image

After unpackaging, you should configure the docker image to be built.

Configuration of Kiuwan docker image is currently done by creating a configuration file (settings.custom) that will be used during the image building process.

There's a file (settings.tpl) you can use as a template to configure your own settings.

  1. Create settings.custom file by copying the template file
    • cp settings.tpl settings.custom
  2. Edit the properties of settings.custom file

 

 

 

PropertyName

Default value

Meaning

SysAdmin configuration
KIUWAN_HOST
KIUWAN_PORT

kop.mydomain.com
7080

Hostname (or ip address) and port number to access KOP.

These variables will be used to build Kiuwan URL.

Example (with default values):

  • http://kop.mydomain.com:7080/saas
KIUWAN_DOCKER_IMAGE Name of the docker image (repository name) to be built

DIR_PERSISTENT_VOLUME

/Datavol

Root directory of docker server where the container volume will be bound

KIUWAN_VOL/kiuwan_vol

Directory of the container where KOP will be installed.

EXTERNAL_PORT_TO_MAP_MYSQL_INTERNAL_PORT3306

Docker server port to access container's MySQL

  • you can leave it to standard 3306 if MySQL is not running in server host
EXTERNAL_PORT_TO_MAP_SSH_PORT2222

Docker server port to access container thorugh SSH

  • must be different to standard 22 because most probably SHH:22 will be running in server host
MONGODB_STARTfalse

Depending of your Kiuwan On Premises purchased license:

  • true: Start mongodb (license with Architecture included)
  • false: Don´t start mongodb (license without Architecture)
Proxy settings  
USE_PROXYfalse
  • false: don´t use proxy to build container
  • true: use proxy to build container
PROTOCOL_PROXYhttphttp or https
HOST_PROXY
PORT_PROXY
 Hostname (or ip address) and port number of proxy used for Intenet connectivity during the image building process
USER_PROXY
PWD_PROXY
 

Username and password to connect Proxy

KOP users
KIUWAN_USER
KIUWAN_GROUP
kiuwan
kiuwan
System user/group for KOP installation and execution
MYSQL_ROOT_PWDrootPassword for root user of KOP's MySQL instance
KIUWAN_MYSQL_USER
KIUWAN_MYSQL_USERPWD
kiuwan
kiuwan

Username and password used by KOP to access MySQL database

SSH_ROOT_PWDpasswordPassword for root user (SSH) of Kiuwan Container
KIUWAN_EMAIL_ADMIN_ACCOUNT 

Email address of kiuwanadmin user

  • this email address will be used to send notifications to kiuwanadmin user
KOP e-mail (SMTP) configuration

KIUWAN_SMTP_HOST
KIUWAN_SMTP_PORT

 SMTP host and port
KIUWAN_SMTP_USER_ACCOUNT
KIUWAN_SMTP_USER_PASSWORD
 SMTP account : username and password
KIUWAN_EMAIL_USER_ACCOUNT 

Sender email address for KOP email notifications

Advanced configuration (WildFly, MySQL, Redis, etc)

INNODB_BUFFER_POOL_SIZE 2G 
APPS_JVMMAXMEMORY 3G

Max ammount of heap memory to be used by WidlFly

KIUWAN_REDIS_MAXMEMORY 2gb

Max memory limit to be used by Redis server (https://redis.io/topics/config)

 

You can visit Advanced Configuration for additional configuration parameters.

 

Step 3. Build the Kiuwan Docker image

After configuration, you will build the Kiuwan docker image.

 

IMPORTANT: Please be sure you have already installed KOP licenses, otherwise the docker image will not be built !!

 

To build the docker image:

     cd $KOP_INSTDIR/vX.Y
     ./create_and_init_kop.sh

 

This step requires Internet connectivity.

If your installation uses an proxy, you should have previolsy configured the proxy settings.

 

In case you need to execute the Kiuwan container in a host server without Internet connectivity:

  1. Build the image in a connected server host
  2. Save the image, transfer it to the target host server and then load it and execute it (as further described)

 

 

After image creation, you will be able to list the images in your docker server and identify your kiuwan image by issuing the next command:

     docker images

 

Step 3.1 (Optional) Moving the image to another server host

After creating the image, if you want to move it to another host server you first must save it to a tarball by executing: 

     docker save kop:v1.2 > kop_v12_saved_image.tar 
Then,  transfer the image tarball to the destination host and load it by executing:
     docker load --input kop_v12_saved_image.tar 

 

Step 4. Executing the Kiuwan Docker container

After executing docker image build script (./create_and_init_kop.sh ), two scripts are generated based on settings.custom configuration file

 

    run_kop.sh 

Script to start the Kiuwan docker container and make KOP fully available and accessible to users.

 

    run_init_kop.sh 

Script to initilize the Kiuwan docker container (i.e. resetting completely the KOP installation to factory defaults) without to create a new image. This script will remove any existing data of KOP installation (for example, all the database contents will be deleted).

 

IMPORTANT:

  • These scripts are based on settings.custom values at the momment of building the docker image.
  • If you later change settings.custom file, the new configuration values will not be applied until a new image is built (and new scripts will be generated)

 

 

You can stop the Kiuwan container by issuing the following command:

     docker stop <my_container_name>

 

 

Step 4.1 Execution in debug mode (only sshd) 

Just in case you are running with problems while executing Kiuwan container, you can run it in debug mode.

This means to apply the configuration but not starting the services, allowing you to access the container through ssh.

 

After the container is built, you can execute (run) the Kiuwan container in debug mode by issuing the following command:

 

     docker run --rm --name <my_container_name> \
                     -h <my_container_host_name> \
                     -v <server_host_mount_dir:container_mount_dir> \
                     -p <ssh_port_ext>:22  \
                     -d \
                     <image_name:version>

 

Step 5. (Optional) Making Kiuwan On Premise running on HTTPS

 

In you need to execute Kiwuan On Premise over HTTPS protocol, please follow next steps. 

Step 5.1 Create your KOP SSL configuration file

SSL configuration  is currently done by creating a configuration file (ssl.custom) within ssl directory 

There's a file (ssl.tpl) you can use as a template to configure your own settings.

  1. cd ssl
  2. Create ssl.custom file by copying the template file
    • cp ssl.tpl ssl.custom
  3. Edit the properties of ssl.custom file and change default values

 

Property Name

Default value

Meaning

Commons
SSL_O Your Organization

SSL_LOCALITY

 

Your Locality

SSL_STATE Your State
SSL_COUNTRY Your Country
SSL_OU Your Organization Unit
Keystore
SSL_KS_PWD Password for Keystore that will be created
Aliases
SSL_ALIASwildfly

Alias of the Certificate to be created.

HTTPS_PORT443HTTPS port

 

Step 5.2 Create the Private key and Certificate Signing Request (CSR) for your server

  1. within ssl directory execute the script CreateKey_and_ReqCSR.sh 
  2. this script generates the CSR file under ssl/certs
    • that file is named <yourhost.yourdomain.com>.csr , according to  $KIUWAN_HOST configuration property
  3. send CSR file to your CA (Certificate Authority) 
  4. CA will send back to you two files: 
    • CA's Certificate file (IMPORTANT: rename it to ca.crt )
    • your host's Certificate file : for example yourhost.yourdomain.com.crt
  5. copy received files to ssl/certs directory

Step 5.3 Create the Keystore and switch from HTTP to HTTPS

  1. within ssl directory execute the script TransferFilesToContainer.sh
    • this script transfers your server's certificate, your private key and CA's certificate to KOP container
    • also, it transfers the script templates that will be used to create the keystore and to change the configuration from http to https 

  2. within ssl directory execute the script run_create_Keystore.sh 

    • this script executes (into the container) the script create_Keystore.sh (created from template create_Keystore.tpl)
  3. within ssl directory execute the script run_change_ToHTTPS.sh 
    • this script stops wildfly service and executes the script change_ToHTTPS.sh (created from template change_ToHTTPS.tpl), this script will create files with .rollback extension of modified ones
    • then, it starts wildfly service 

Step 5.4 Just in case you are using your own CA, make it valid to your browsers and Java 

If the certificate is signed by your own Certification Authority, the browsers will not recognize it as a valid CA and you will get an error messages such as:

Your connection is not private
Attackers might be trying to steal your information from youthost.yourdomain.com (for example, passwords, messages, or credit cards).

NET::ERR_CERT_AUTHORITY_INVALID


To solve this issue, you have to import your CA Certificate into your browser

  • In Chrome: Configuration >> Settings >> Advanced >> Manage certificates >>  Import (ca.crt) into "Trusted root certification entity store"

 

Additionally, you must import your CA Certificate into Java's Keystore (cacerts):

Log in to the KOP container and execute next commands:

  1. cd /opt/jdk1.8.0_141/jre/lib/security/
  2. supervisorctl stop wildfly 
  3. cp cacerts cacerts.bck.original 
  4. keytool -import -noprompt -alias root -keystore cacerts -trustcacerts -file /<kiuwan_vol>/configurations/ssl/ca.crt -storepass changeit
    • NOTE: change <kiuwan_vol> to the value of $KIUWAN_VOL (as configured in settings.custom
  5. supervisorctl start wildfly

Then, log in to your docker server and make above changes persistent (i.e. to keep them after rebooting the container):

  1. docker commit <kop docker name> <kop docker image> 
    • NOTE: run 'docker ps' to get NAMES and IMAGE values of your KOP container

 

Rollback to HTTP

In case you need to change back to HTTP, follow next steps:
  1. Log in to your docker server and go to your installation directory ($KOP_INSTDIR)
  2. cd ssl
  3. execute the script run_rollback_HTTPS.sh 

After execution, KOP will come back to configuration previous to the execution of run_change_ToHTTPS.sh script.

 

Step 6. Accessing Kiuwan On Premise

KOP URL

Once Kiuwan On Premise container is running, you can access it from a browser in the following URL:

http[s]://<KIUWAN_HOST>:<KIUWAN_PORT>/saas 

where KIUWAN_HOST and KIUWAN_PORT match the values of those properties as configured in settings.custom


KOP built-in users

KOP comes with the following built-in users.

  • kiuwanadmin (password: kiuwanadmin)
    • access to Kiuwan "functional" administration modules such as Users, Applications and Model Management  (see Admin Guide )
    • log in as kiuwanadmin to create users of your KOP instance
  • sysadmin (password: sysadmin)
    • log in as sysadmin to access functionalities related to monitoring and tuning KOP execution
    • sysadmin will give you access KOP sysconsole


Upgrading your KOP installation

In order to maintain your KOP installation up-to-date to new features, bugfixes, etc, you will have to upgrade your KOP installation.

The following pages describe the upgrading Installation procedures of releases of KOP.

 

Basically, a new relase is distributed as a new KOP Installation Package,  consisting on a tarball gz file (docker-for-kiuwan_master.X.Y.tar.gz) containing all the new kiuwan docker files.

  1. copy the new distribution tarball to your host server’s updates directory ($KOP_INSTDIR/updates/download)
  2. uncompress the distribution tarball  
    • tar xvzf docker-for-kiuwan_master.X.Y.tar.gz 
    • $KOP_INSTDIR/updates/download/docker-for-kiuwan_master.X.Y directory  will be created
  3. change directory to $KOP_INSTDIR/updates/bin
  4. execute the script updateKiuwanLauncher.sh
    • this script makes all the work to upgrade your current KOP app instance to the new one
  5. execute the script updateClientsLauncher.sh
    • this script makes new KLA and K4D ready to be downloadable by KOP client users

 

 

Advanced Configuration


As above specified, settings.custom file allows you to specify most (but not all) configuration parameters.

There are some additional configuration parameters that you can configure at Dockerfile (located in your $KOP_INSTDIR)

 

Configuration done at settings.custom file always take precedence over configuration done at Dockerfile 



Component

PropertyName (=default value) 

Meaning

TimeZoneKIUWAN_TIMEZONE='UTC'Timezone to be used by Kiuwan application.
WildFly

APPS_JVMMINMEMORY=512M
APPS_JVMMAXMEMORY=3G

Max and Min ammount of heap memory to be used by WidlFly.
APPS_JVMMINMETASIZE=96M
APPS_JVMMAXMETASIZE=512M

Max and min ammount of perm gen memory to be used by WildFly

(see https://issues.jboss.org/browse/WFLY-5955)

MySQL

innodb_buffer_pool_size=2G
join_buffer_size=4M
query_cache_limit=16M
read_buffer_size=16M
read_rnd_buffer_size=4M
sort_buffer_size=4M
max_connections=1024
tmp_table_size=16M
max_allowed_packet=64M

Default MySQL parameters used to create database instance.

(see https://dev.mysql.com/doc/refman/5.6/en/ for reference) redis

Redis

KIUWAN_REDIS_MAXCLIENTS=100
KIUWAN_REDIS_MAXMEMORY=2gb

Maximum number of Redis clients that could be handled simultaneously (https://redis.io/topics/clients)

Max memory limit to be used by Redis server (https://redis.io/topics/config)

  • No labels