...
...
| Page Tree | ||
|---|---|---|
|
...
...
...
...
...
...
...
...
A typical role distribution
KOP solution is deployed on a centralized server (all the components are installed in the same host) and accessed by users via HTTP website access or by IDE plugins (Eclipse, Visual Studio).
===============
System Architecture Overview
Kiuwan On-Premises (KOP) is based on a client-server architecture with the following major server components:
- WildFly application server
- MySQL database
KOP solution is deployed on a centralized server (all the components are installed in the same host) and accessed by users via HTTP website access or by IDE plugins (Eclipse, Visual Studio).
KOP also supports distributed architecture with high-availability and load-balancing. For these scenarios, please contact Kiuwan support.
Server Host Requirements
Kiuwan On-Premises (KOP) solution is deployed on a centralized server host (all the components are installed in the same host) and accessed by users via HTTP or by IDE plugins (Eclipse, Visual Studio).
| Info | ||
|---|---|---|
| ||
|
Server CPU and memory requirements depend on several factors:
- how many lines of code will need to be scanned
- how many analyses will need to be executed in parallel
- how many users will access the web application
| Info |
|---|
It's important to notice that we're assuming the typical use case where code analyses are performed locally through Kiuwan Local Analyzer, and therefore out of server machine (tipically at a user machine or within a build CI system). Therefore, when we talk about analysis we mean 2nd phase of the analysis (i.e. indicator calculation that is executed within the host server) |
Above factors heavily influence host server requirements, but minimum requirements are:
| Operating System | RAM | CPUs | Disk | Other SW |
|---|---|---|---|---|
Linux kernel version 3.10 or higher | 16 GB | Intel i7 2,5 GHz with 4 cores | 750 GB | CS Docker Engine version 1.10 or higher |
Note: CPU clock speed and disk speed will affect overal response time.
Above configuration is recommended for a system load that requires continuous service for:
- parallel execution of 2 analyses (any additional parallel analysis request will be enqueued, and it will be executed as soon as any of the running analyses finishes), and
- 50 concurrent web-users
Additional system requirements can be met by following next recommendations:
- In case you need a higher number of parallel analyses executions, you should add 1 CPU core and 512 Mb for any additional required parallel analysis
- In case you need to improve response time to web users, conside to add 1Gb for every 50 additional web users.
Installation steps
Before installing KOP, make sure that you understand the System Architecture Overview, that your server host(s) complies with the Server Host Requirements, and that you have properly prepared the Environment as follows:
...
Installation requirements:
The installation requires CS Docker Engine version 1.10 or higher installed on your server
Be sure you meet Server Host Requirements
The first phase of installation process (building the Kiuwan image) requires Internet connectivity
Installation must be executed using any unix user with privileges to execute docker server commands
| Table of Contents | ||
|---|---|---|
|
Step 1. Unpackage KOP Installation Package
| Info |
|---|
KOP Installation Package consists on a tarball gz file (vX.Y.tar.gz) containing all the kiuwan docker files.
KOP Licenses are distributed through two license distribution zipfiles:
To install the licences:
|
Step 2. Configure Kiuwan Docker image
...
After unpackaging, you should configure the docker image to be built.
Configuration of Kiuwan docker image is currently done through below properties.
This configuration can be specified in two ways:
...
PropertyName (=default value) | Meaning |
|---|---|
KIUWAN_VOL=/kiuwan_vol | Directory of the container where Kiuwan software will be installed. |
| SSH_ROOT_PWD=password | Password for root user (SSH) of Kiuwan container |
MYSQL_ROOT_PWD=root | Password for MySQL root user when creating Kiuwan database. |
KIUWAN_MYSQL_USER=kiuwan | Username and password used by Kiuwan app to access MySQL database |
KIUWAN_HOST=kiuwan.mydomain.com | Hostname (or ip address) and port number to access Kiuwan. These variables will be used to build Kiuwan URL. Example (with default values):
|
KIUWAN_EMAIL_ADMIN_ACCOUNT | Email address of Kiuwan admin user
|
KIUWAN_EMAIL_USER_ACCOUNT | Sender email address for Kiuwan notifications
|
KIUWAN_SMTP_USER_ACCOUNT | SMTP account : username and password |
KIUWAN_SMTP_HOST= | SMTP host and port |
KIUWAN_WILDFLY_USER=kiuwan | System user/group for WildFly's installation and execution |
You can visit Advanced Configuration for additional configuration parameters.
Step 3. Build the Kiuwan Docker image
| Info |
|---|
After configuration, you will build the Kiuwan docker image.
IMPORTANT: Please be sure you have already installed KOP licenses, otherwise the docker image will not be built !!
To build it (configuration done at Dockerfile): cd $KOP_INSTDIR/vX.Y docker build -t <image_name:version> .where image_name can be any string that helps you to identify the image (for example kop:v1.2)
If you prefer to specify the configuration by command line arguments, you can use: docker build \ --build-arg KIUWAN_HOST=mykiuwan.mydomain.com \ --build-arg KIUWAN_PORT=myPortNumber \ --build-arg ... \ -t <image_name:version> .
This step requires Internet connectivity. If your installation uses an proxy, you should build it with next command:
docker build \ --build-arg http_proxy=http://user:pass@proxy_host:proxy_port \ -t <image_name:version> .In case you need to execute the Kiuwan container in a host server without Internet connectivity:
|
After image creation, you will be able to list the images in your docker server and identify your kiuwan image by issuing the next command:
docker images
Step 3.1 (Optional) Moving the image to another server host
After creating the image, if you want to move it to another host server you first must save it to a tarball by executing:
docker save kop:v1.2 > kop_v12_saved_image.tar docker load --input kop_v12_saved_image.tar
Step 4. Post-build configuration of the Kiuwan Docker image
| Info |
|---|
After building the image, an initial execution of the image it’s need to complete the installation process. To do it, execute: docker run --rm --name <my_container_name> \ -h <my_container_host_name> \ -v <server_host_mount_dir:container_mount_dir> \ -e KIUWAN_FIRST_TIME="Y" -it \ <image_name:version>
This step will configure and populate Kiuwan database as well as set up directory structure of Kiuwan image. This step only should be executed once and previously to run the container. |
For example, the next command:
docker run --rm --name kop -h kop.mydomain.com -v /data/kop:/kiuwan_vol -e KIUWAN_FIRST_TIME="Y" -it kop:v1.2- will assign "kop" as the name of the container
- will assign "kop.mydomin.com" as the hostname of the container
- will install KOP sw in
- /data/kop in server host
- /kiuwan_vol/ in container
Step 5. Executing the Kiuwan Docker container
...
After the container is built, you can execute (run) the Kiuwan container by issuing the following command:
docker run --rm --name <my_container_name> \ -h <my_container_host_name> \ -v <server_host_mount_dir:container_mount_dir> \ -p <mysql_port_ext>:3306 -p <ssh_port_ext>:22 -p <kiuwan_port_ext>:7080 \ -e START_AND_RUN="Y" -d \ <image_name:version> - <container_mount_dir> must match KIUWAN_VOL property of Dokerfile
You need to map several ports from the container to the host machine in order to access the container. That’s done by using –p server_port:container_port.
- <mysql_port_ext> : server host port to access container's MySQL (you can leave it to standard 3306 if MySQL is not running in server host)
- <ssh_port_ext> : server host port to access container thorugh SSH (must be different to standard 22 because most probably SHH:22 will be running in server host)
- <kiuwan_port_ext> : external port to access Kiuwan ( IMPORTANT: <kiuwan_port_ext> must match KIUWAN_PORT property of Dokerfile)
You can stop the Kiuwan container by issuing the following command:
docker stop <my_container_name>This System Administration Guide is addressed to the sysadmin user.
Please refer to Kiuwan Admin Guide for the Kiuwan functional administration guide.
SysAdmin Console
| Info | ||
|---|---|---|
| ||
The system administration user (sysadmin) has access to the SysAdmin Console : http://<$KIUWAN_HOST>:<$KIUWAN_PORT>/saas Please note that the SysAdmin console uses the same URL that KOP URL, just login in as sysadmin and SysAdmin Console will appear. |
Sysadmin console provides access to following functionalities:
- System Administration page
- Account Management
- Analysis Administration
- Insights Administration
- Support
System Administration page is the default page when you log in to SysAdmin Console.
All the other options are available through the drop-down menu.
System Administration
Once you log into SysAdmin Console, the System Administration page is displayed
System Administration page display the following contents:
- Licensed products
- A list with licensed products for your KOP installation
- License expiration date
- The expiration date for your KOP license
- Kiuwan Core
- Installed version and Last available version for Kiuwan Core
- In case your installed KOP version is not the last available, you may decide to upgrade.
- For instructions on upgrade your KOP installation, please visit Upgrading your KOP installation
- Kiuwan Clients
- Installed version and Last available version for each of Kiuwan Clients ( Kiuwan Local Analyzer, Kiuwan Engines and Kiuwan 4 Developers for supported IDE's)
- In case of any installed KOP client is not the last available, you may decide to upgrade.
- For instructions on upgrade your KOP installation, please visit Upgrading your KOP installation
- Minimum KOP version needed to upgrade clients
- There are some dependencies between Kiuwan Clients and Core.
- This field indicates the minimum KOP Core version needed to be installed in order to upgrade the KOP Clients.
- If your KOP current version is older than the minimum required, you need to upgrade Kiuwan Core.
Account Management
The Account Management page lets you change the password for the sysadmin user.
Analysis Management
The Analysis Management page lets you access the Queues Monitor
A complete Kiuwan analysis involves two-phases:
- 1st Phase - Local Analysis
- KLA analyzes source files and upload reports to Kiuwan
- 2nd Phase - Cloud Metrics Calculation
- Indicators and metrics are calculated in the cloud based on uploaded analysis reports
KOP uses an in-memory message queue to process those reports. This message-based queue ("Reports generated queue") allows to process (and manage) analyses in an orderly fashion
Messages in this queue are processed by a JVM (executing WildFly and KOP application).
By default, the queue is attended by one thread in the KOP JVM.
This means that, by default, analyses are sequentially processed.
States of an analysis
Once the local analysis finishes, KLA uploads reports to KOP and identifies the analysis by an Analysis Code (e.g. A-7e2-1669fcae0cf).
Then, the analysis could be in either of two possible states:
| Analysis State | Meaning |
|---|---|
| STATIC_ANALYSIS_FINISHED | (1) The local static analysis is finished and results are waiting to be processed (to calculate indicators) |
PARSING_REPORTS | (2) The analysis reports are currently being processed (in progress) |
Messages in the queue
Queue monitor displays the number of messages in the Reports Generated Queue.
This means the number of analyses waiting to be processed.
If this number is constantly greater than 0, that means that there's a bottleneck and you (as sysadmin) should increase the throughput to avoid waiting states.
As said above, by default the KOP JVM listens to the message queue with 1 thread. You can configure the max number of threads by introducing an upper value and click on the Change button.
Also, you can stop processing by clicking on the Stop button.
Active Analyses
Click Active Analyses and the queue monitor will display ALL the active analyses, regardless of their state (STATIC_ANALYSIS_FINISHED and PARSING_REPORTS).
REPORTS_GENERATED_QUEUE
Click REPORTS_GENERATED_QUEUE, the queue monitor will display only the analyses waiting to be processed (i,e STATIC_ANALYSIS_FINISHED).
Dispatch selected Analysis
As said before, messages queues are maintained in memory (in Redis). But, message sates are also maintained in ddbb (MySQL).
Sometimes, Redis and MySql might get unsynced. This is not frequent, but after some JVM crashes (for whatever reason) you could get this situation.
The most frequent unsyncing situation might be that an analysis is in PARSING_REPORTS state (in MySql) but it's not currently being processed (you will see that is not "red" in Active Analyses table.
This would mean that analysis is "orphaned".
In this case, the analysis needs to be manually enqueued. To do it, just check the analysis and click Dispatch selected analysis. This action will re-enqueue the analysis.
Support
The support page allows collecting the most important log files of KOP installation for troubleshooting purposes.
Click Extract support data to obtain a zip file with the most relevant log files.
Step 6.1 Execution in debug mode (only sshd)
Just in case you are running with problems while executing Kiuwan container, you can run it in debug mode.
This means to apply the configuration but not starting the services, allowing you to access the container through ssh.
| Info |
|---|
After the container is built, you can execute (run) the Kiuwan container in debug mode by issuing the following command:
docker run --rm --name <my_container_name> \ -h <my_container_host_name> \ -v <server_host_mount_dir:container_mount_dir> \ -p <ssh_port_ext>:22 \ -d \ <image_name:version> |
Step 6. Accessing Kiuwan On Premise
| Info |
|---|
Once Kiuwan On Premise container is running, you can access it from a browser in the following URL: http://<$KIUWAN_HOST>:<$KIUWAN_PORT>/saas where KIUWAN_HOST and KIUWAN_PORT match the values of those properties as configured in Dockerfile |
Advanced Configuration
Below table shows available configuration parameters :
...
PropertyName (=default value)
...
Meaning
...
KIUWAN_WILDFLY_JVMMINMEMORY=512M
KIUWAN_WILDFLY_JVMMAXMEMORY=3G
...
Max and min ammount of perm gen memory to be used by WildFly
(see https://issues.jboss.org/browse/WFLY-5955)
...
innodb_buffer_pool_size=2G
join_buffer_size=4M
query_cache_limit=16M
read_buffer_size=16M
read_rnd_buffer_size=4M
sort_buffer_size=4M
max_connections=1024
tmp_table_size=16M
max_allowed_packet=64M
...
Default MySQL parameters used to create database instance.
(see https://dev.mysql.com/doc/refman/5.6/en/ for reference) redis
...
KIUWAN_REDIS_MAXCLIENTS=100
KIUWAN_REDIS_MAXMEMORY=2gb
Maximum number of Redis clients that could be handled simultaneously (https://redis.io/topics/clients)
...