Table of contents
- Overview
- Getting started
- Configuration
- Examples
- Use older eShop version
- Use NFS for shared-folder
- Change PHP version
- Change MySQL version
- Change VM provider
- Set eShop to UTF-8 mode
- Use SSL
- Change shared folder path
- Define github token for composer
- Change ubuntu repository mirror url
- Change virtual host
- Change the display mode of errors
- Change MySQL password
- Trigger Varnish installation
- Trigger Selenium installation
- Trigger IonCube integration
- Customize email monitoring integration
- Customize MySQL administration web app integration
- Composer parallel install plugin
- Examples
- SDK
- Guides
- Troubleshooting
- Provision process hangs on "Run composer install" task
- Error from Unit_Admin_ModuleListTest::testRender() method while testing eShop
- Browser shows "Zend Guard Run-time support missing!"
- On Windows machines, getting "requires a TTY"
- Selenium tests do not run after VM was destroyed: error "Session not started"
- Composer returns "ProcessTimedOutException"
- On Windows machines, fails to install vagrant-hostmanager plugin
Current OXID eShop development environment is inspired by PuPHPet and Phansible projects.
Virtual environment is built using:
- Vagrant - virtual environment automation tool;
- Ansible - environment orchestration tool;
- YAML - solution configuration.
Final solution is composed of two repositories (linked using git sub-modules):
- Base VM - Base LAMP stack (also used as base for other VMs);
- eShop VM - Current repository, eShop specific configuration, roles and SDK components.
Before proceeding with Quick start please ensure that the Dependencies listed below are installed.
- Vagrant (>=1.8.6)
- VirtualBox [1] (>=4.2, except 5.1.8, see #29; Windows users see #32)
- Git
- OpenSSH (Only client part is needed)
- Vagrant plugins:
Note for Windows users: Use console with Administrator privileges to run vagrant commands!
- Clone [2] out current repository:
git clone --recursive https://github.com/OXID-eSales/oxvm_eshop.git
- Start the VM:
cd oxvm_eshop
vagrant up
- After successful provision process use the following links to:
- Open OXID eShop: http://www.oxideshop.dev/
- Access admin area: http://www.oxideshop.dev/admin/
- Username:
admin
- Password:
admin
- Username:
- If something doesn't work, see the Troubleshooting section.
[1] | VirtualBox is listed as dependency due to the fact that it is the default chosen provider for the VM. In case other providers will be used there is no need to install VirtualBox. Please refer to the list of possible providers in the configuration section to get more information. |
[2] | Since the current eShop VM repository is linked through git sub-modules
it is mandatory to use --recursive option to instruct git and clone
base VM repository as well. |
Default virtual environment configuration should be sufficient enough to get the eShop CE/PE/EE versions up and running. However, it is possible to adjust the configuration of virtual environment to better match personal preferences.
All configuration changes should be done by overriding variables from:
- default.yml - base vm variables;
- oxideshop.yml - eShop specific variables.
These overridden values must be placed in personal.yml
[3] file at the root level of current repository.
_
For the overridden values to take effect please run vagrant provision
. If
the changes are related to the shared folder use vagrant reload
. In case the
provision process will start to show any kind of errors, please try to use
vagrant destroy && vagrant up
for the process to stat over from a clean
state.
To double check the merged version of configuration just run vagrant config
.
Hint: you have to care for the syntax/semantics at yourself. So, if you get an error while vagrant provision
your personal.yml is the start point for troubleshooting. Has every entry value? At the moment no empty entries will work.
Below is a list of possible frequent changes which are typically done after cloning this repository.
One can just copy and paste the example snippets from the list below to the
personal.yml
file at the root of this repository.
The current VM configuration is optimized to work with latest version of OXID eShop: v6.0-beta
.
In order to use VM with an older OXID eShop version please make the following adjustments to the configuration
file:
---
eshop:
composer:
vendor: "{{ eshop_path.source }}/vendor"
Don't forget to follow the installation instructions which are provided with the eShop,
e.g. version 5.3
requires some additional steps before the provision process. These steps might include (but is not limited to) addition of flow theme and demodata files.
If LXC is not available on your host system the best (so far) method to share your application folder would be to use NFS.
In order to trigger NFS usage inside the VM one has to add the following configuration:
---
vagrant_local:
vm:
app_shared_folder:
sync_type: nfs
Beware that in order for this to work your host system must support NFS server:
- OS X/macOS has this integrated out-of-the-box
- Various GNU/Linux distributions might require additional setup:
Be aware that NFS also has it's own limitations:
- No server is available for Microsoft Windows
- NFS uses network stack to share data
- NFS does not propagate file change events to the guest system
- Sometimes NFS is complicated to setup/troubleshoot in a non-standard environment
By default latest PHP version found in ubuntu repository is installed.
When PHP version is specified, PHPBrew is installed and used for switching between versions. Requested version will be either built on the fly or downloaded from assets [4] repository.
---
php:
version: 5.3
Keep in mind that by default this setting only affects the CLI interface of PHP, in order to change the PHP version for Apache, please apply the following additional commands:
sudo cp /etc/apache2/mods-available/php5.6.conf /etc/apache2/mods-available/php5.conf
sudo a2dismod php5.6
sudo a2enmod php5
phpswitch 5.3 # or 5.4, 5.5, 5.6, 7.0, 7.1
To disable downloading of cached versions from assets repository, set cache_repository
to empty value.
Alternatively it is possible to build your own PHP packages and place them into any svn repository.
Only when php version is specified, PHPBrew will be installed so those commands became available inside VM:
phpbrew list
- lists installed PHP versionsphpbrew update --old
- Updates PHP versions list with old php versionsphpbrew known
- lists available PHP versionsphpbuild [version]
- builds PHP versionphpswitch [version]
- switch PHP versionphpswitch off
- switch back to default PHP version
When versions is downloaded from assets repository, phpbrew will not have its source code and therefore will not be able to build php extensions. To download PHP source run this command with full php version specified:
phpbrew download [phpversion] && tar jxf ~/.phpbrew/distfiles/php-[phpversion].tar.bz2 -C ~/.phpbrew/build/
MySQL versioning is not yet automated via Ansible, in order to change the
version of MySQL service, please apply the following commands after calling
vagrant ssh
:
wget http://dev.mysql.com/get/mysql-apt-config_0.8.7-1_all.deb
sudo dpkg -i mysql-apt-config_0.8.7-1_all.deb
# Choose MySQL version to install
sudo apt-get update
sudo apt-get install mysql-server
Change VM provider from VirtualBox (default) to LXC. A list of available and tested providers [5]:
- virtualbox - Default provider which is free to use and available on all major operating systems;
- lxc [6] - Operating system level virtualization which vastly improves I/O performance compared to para-virtualization solutions;
- parallels [7] - Commercial VM provider for OS X.
---
vagrant_local:
vm:
provider: lxc
By default shop will be installed with UTF-8 mode disabled (with
iUtfMode = '0'
value inside config.inc.php
).
In order to turn on the UTF-8 mode:
---
eshop:
config:
utf_mode: 1
This change will not affect the already configured shop [8] .
By default the shop is served using regular HTTP protocol. In order to use SSL use the following configuration snippet:
---
eshop:
config:
use_ssl: 1
This configuration will automatically update config.inc.php
file to contain
necessary changes for using SSL.
Change the default application shared folder from oxideshop
to local path
/var/www
and update eShop target folder [9].
---
vagrant_local:
vm:
app_shared_folder:
source: /var/www
target: /var/www
eshop_target_path: /var/www/oxideshop
Provide OAuth token from github for composer so that the access API limit could be removed [10].
---
php:
composer:
github_token: example_secret_token
Change the default ubuntu repository mirror url from http://us.archive.ubuntu.com/ubuntu/
to http://de.archive.ubuntu.com/ubuntu/
.
---
server:
apt_mirror: http://de.archive.ubuntu.com/ubuntu/
Change the default virtual host from www.oxideshop.dev
to
www.myproject.dev
.
---
vagrant_local:
vm:
aliases:
- www.myproject.dev
By default the display_errors option is turned on. To change the behavior you can use:
---
php:
display_errors: Off
Change the default MySQL user password from oxid
to secret
.
---
mysql:
password: secret
Trigger Varnish installation so that it can be used within OXID eShop.
---
varnish:
install: true
The above change will only trigger installation of Varnish with it's distributed
default configuration (default.vcl
) which is not compatible with OXID eShop.
In order to make Varnish compatible with OXID eShop the following items must be done:
- Download
oxid-esales/varnish-configuration
composer package; - Copy the
VCL
files into the system where Varnish is used; - Adapt
servers_conf.vcl
file to match system environment; - Restart Varnish service;
- Update OXID eShop
config.inc.php
; - Update OXID eShop settings in admin area.
The above steps are described with more detail in the topics below:
Because oxid-esales/varnish-configuration
is a composer
package and
composer
tool is available for VM by default we could use the following
OXID eShop version independent way to download the package:
# Register private password protected repository composer global config repositories.oxid-esales/varnish-configuration \ composer https://packages.oxid-esales.com/varnish/ # Download the actual package composer global require oxid-esales/varnish-configuration
Keep in mind that composer
will ask for username and password as the package
is available only to users who have bought the performance package
(https://www.oxid-esales.com/performance/). Please use the credentials which
were provided during the purchase.
By default a configuration which is compatible with Varnish 4.0.x
will be
provided. In order to see which other versions are available as well, please use
the following command:
composer global show --all oxid-esales/varnish-configuration
And to download the package with selected version, e.g. 3.0.0
:
composer global require oxid-esales/varnish-configuration:v3.0.0
After the package has been downloaded into global vendor
directory as
described by the topic above it becomes possible to transfer configuration
files into the system by using these commands:
sudo cp $HOME/.composer/vendor/oxid-esales/varnish-configuration/default.vcl \ /etc/varnish/ sudo cp $HOME/.composer/vendor/oxid-esales/varnish-configuration/servers_conf.vcl.dist \ /etc/varnish/servers_conf.vcl
There are two mandatory placeholders which need to be updated inside the
servers_conf.vcl
file:
<my_shop_hostname>
- a valid host which could be used to communicate with the shop internaly;<my_shop_IP>
- an inbound external IP address which has rights to trigger cache invalidation.
The following commands could be used with a default configuration of VM to replace the placeholder values with suitable ones:
sudo sed -i "s/<my_shop_hostname>/127.0.0.1/g" /etc/varnish/servers_conf.vcl sudo sed -i "s/<my_shop_IP>/$(ip addr | grep eth1 | tail -n 1 \ | grep -oE "(\b([0-9]{1,3}\.){3}[0-9]{1,3}\b)" | head -n 1)/g" \ /etc/varnish/servers_conf.vcl
After adapting the configuration files it's now possible to restart Varnish service in order for the updated configuration to take effect:
sudo /etc/init.d/varnish restart
Because Varnish uses 6081
port by default this needs to be reflected in
the configuration file /var/www/oxideshop/source/config.inc.php
.
If e.g. the hostname was used as oxideshop.dev
the following change must be
applied, from:
$this->sShopURL = 'http://oxideshop.dev';
into:
$this->sShopURL = 'http://oxideshop.dev:6081';
After all of the steps above one must apply necessary changes in the admin area of the OXID eShop:
- Visit http://oxideshop.dev:6081/admin/
- Choose
Master settings
- Select
Core settings
- Switch to
Caching
- Expand
Reverse proxy
- Tick
Enable caching
To check if Reverse proxy cache is active, please click
Test Reverse Proxy's availability
. In case of successful configuration
the following green colored message will appear "Reverse Proxy test succeed".
Trigger Selenium installation so that it can be used to run Selenium tests with the help of OXID testing library.
---
selenium:
install: true
Together with Selenium, a vnc server is installed in order to connect via remote
display. Suitable clients are e.g. vinagre
on Linux or the built-in vnc
client of OS X.
Trigger IonCube integration so that it can be used to decode the encoded files.
---
ioncube:
install: true
Integration of Mailhog allows to monitor e-mail activity from the eShop. List of e-mails could be seen at: http://www.oxideshop.dev/mail/
Possible configuration options for Mailhog:
web_port
- web UI port (8025
is the default value which means that the UI can be accessed by the following URL: http://www.oxideshop.dev:8025/)smtp_port
- SMTP server port (1025
is the default value)web_alias
- an URL alias for the default virtual host to act as a proxy for web UI of mailhog (/mail/
is the default value which means that the UI can be access by the following URL: http://www.oxideshop.dev/mail/)
An example configuration which changes web UI port to 8024
, SMTP port to
1026
and alias to /emails/
:
---
mailhog:
web_port: 8024
smtp_port: 1026
web_alias: /emails/
Mailhog is installed by default as it has install: true
option in the
default configuration file. In order to disable email monitoring please use the
following configuration snippet:
---
mailhog:
install: false
Integration of Adminer allows to access
MySQL administrative tasks and data through web alias adminer
at:
http://www.oxideshop.dev/adminer/
Integration of
Adminer editor allows
to access and modify MySQL data through web alias adminer_editor
at:
http://www.oxideshop.dev/adminer_editor/
Possible configuration options for Adminer and Adminer editor:
pkg_url
- An URL which points to the compiled PHP version of the applicationweb_alias
- An alias used to access the application (Default value isadminer
/adminer_editor
, which means that in order to access it one has to open http://www.oxideshop.dev/adminer/ / http://www.oxideshop.dev/adminer_editor/)pkg_sha256
- A SHA-256 hash of file contents downloaded from resource defined inpkg_url
Adminer and Adminer editor are installed by default as they have
install: true
option in the default configuration file. In order to disable
these tools please use the following configuration snippet:
---
adminer:
install: false
adminer_editor:
install: false
Keep in mind that your MySQL credentials will be already present in the login page and there is no need to enter the login data manually. The following variables are used to gain MySQL credentials:
mysql.user
- User name which has access to the created databasemysql.password
- Password of previously mentioned usermysql.database
- Name of the newly created database
The composer parallel install plugin hirak/prestissimo is enabled by default. In order to disable it please use the following snippet:
---
php:
composer:
prestissimo:
install: false
[3] | personal.yml configuration file is already
included in .gitignore and should not be visible as changes to the actual
repository. |
[4] | Repository with some already prebuilt versions of php for faster installation. |
[5] | VM solutions from VMWare, such as workstation and fusion were not yet adapted or tested with our current configuration of VM. |
[6] | Keep in mind that LXC provider is only available for
GNU/Linux based operating systems. In order to start using this provider with
vagrant a plugin must be installed for it
(vagrant plugin install vagrant-lxc ). So far it has been only tested with
Ubuntu based OS with lxc package installed (sudo apt-get install lxc ). |
[7] | A vagrant plugin must be installed
(vagrant plugin install vagrant-parallels ) in order to use vagrant with
Parallels. |
[8] | Keep in mind that the provided snippet will not change
the UTF-8 mode of the eShop if the configuration file (config.inc.php ) is
already present and defined. In this case one has to make the change of
iUtfMode value directly in the config.inc.php file of the eShop. |
[9] | Keep in mind that if the shared folder target does not match
actual application (eShop) target it has to be specified explicitly by
defining eshop_target_path . |
[10] | By default github has API access limits set for anonymous access. In order to overcome these limits one has to create a github token, which could be done as described in: https://help.github.com/articles/creating-an-access-token-for-command-line-use/ |
Out of the box the VM is equipped with the following SDK components:
- Module skeleton generator - module which helps to create new OXID eShop modules;
- Module certification tools - a collection of tools which allows one to see a detailed report from module certification process;
- Testing library - a library for writing various kind of tests inside eShop and a set of tools for running those tests.
There are also other SDK components which could be found at: http://wiki.oxidforge.org/SDK
By default this module is installed under eShop's modules
directory (by
default it will be /var/www/oxideshop/source/modules/
which is defined by
eshop_path.modules
key in configuration).
The module needs to be activated manually. Further instructions on how to enable and use the module could be found at (installation part can be skipped): https://github.com/OXID-eSales/module_skeleton_generator#usage
By default the tools are installed under VM's home folder (~/eshop_sdk
which
is defined by eshop.sdk.path
key in configuration). The repository of tools
is cloned out in ~/eshop_sdk/module_certification_tools
and an extra
shortcut ox_cert
is created inside ~/eshop_sdk/bin/
(it's included in
PATH
environment variable automatically).
There is no need to do any installation part for tools to work as it is already done by the VM's provision process.
In order to invoke the certification report generator just use the provided shortcut:
ox_cert <vendor_name>/<module_name>
An example of invoking the reporting tool for module generator [11]:
$ ox_cert oxps/modulegenerator
After the execution it will generate a HTML document which will be placed at
~/eshop_sdk/module_certification_tools/result/<datetime>/report.html
.
Once the report is generated one can just view the contents of it straight from inside the VM using command line tools or copy the file to shared folder and view it from host machine, e.g.:
cp ~/eshop_sdk/module_certification_tools/result/20150916101719/report.html \
/var/www/oxideshop
Library needed for various testing purposes is already installed in the VM
through the help of composer, because it's defined
in composer.json
as development requirement inside eShop (at least in CE
latest development version).
All binary tools are installed inside /var/www/oxideshop/source/vendor/bin/
(this value may be changed through eshop_path.source
key in configuration).
A list of available binary tools:
reset-shop
- restore eShop's database to it's default state (demo);runmetrics
- run pdepend against eShop and modules code to collect various code metrics information;runtests
- run unit/integartion tests against eShop and modules code;runtests-coverage
- generate coverage report by running unit/integration tests;runtests-selenium
- run acceptance tests written for Selenium.
More details on how to use and configure the library could be found at: https://github.com/OXID-eSales/testing_library
[11] | The tools can be invoked from any working directory as
long as the ox_cert shortcut is being used. |
List of guides for working with VM:
- Open VM directory:
cd oxvm_eshop
- Destroy old VM:
vagrant destroy
- Update eShop VM:
git pull
- Update Base VM:
git submodule update --recursive
- Start VM:
vagrant up
Our current eShop packages have different file/directory structure compared to eShop repositories. VM is suited to work for eShop repository file/directory structure (because it holds our source and test files at the same place). Due to this reason one would need to manually extract eShop source and test packages into shared folder.
In order to prepare VM for usage of eShop packages please follow the steps below before running the VM:
- Create shared folder [12] (By default it's
oxideshop
folder); - Extract source package into
oxideshop/source
folder; - Extract tests package into
oxideshop/tests
folder (This step is optional for eShop runtime).
If the above steps were done after the creation/provision of VM please make
sure to re-run vagrant provision
command for provision process to make
necessary changes.
Don't forget that packages usally contain older version of eShop which requires additional work besides the work which will be done by the provision process.
Directory structure overview of eShop packages placed for the VM:
oxvm_eshop - root of oxvm_eshop repository
+ <oxvm_eshop repository files>
+ oxideshop - shared folder
+ source
+ <eShop source package contents>
+ tests
+ <eShop tests package contents>
An example of commands to prepare VM for using eShop packages:
Keep in mind that the below example only demonstrates how one should prepare the VM for source and test packages. In order to actually get/download source and test packages for eShop PE/EE versions please contact OXID eSales support.
In case the following two packages were received:
OXID_ESHOP_EE_5.2.5_for_PHP_5.6.zip
- source packageOXID_ESHOP_TESTS_EE_5.2.5_for_PHP_5.6_SOURCE.rar
- test package
# Clone out VM repository
git clone --recursive https://github.com/OXID-eSales/oxvm_eshop.git
cd oxvm_eshop
# Download packages
wget http://<url provided by support>/OXID_ESHOP_EE_5.2.5_for_PHP_5.6.zip
wget http://<url provided by support>/OXID_ESHOP_TESTS_EE_5.2.5_for_PHP_5.6_SOURCE.rar
# Extract packages
mkdir oxideshop
unzip OXID_ESHOP_TESTS_EE_5.2.5_for_PHP_5.6_SOURCE.rar -d ./oxideshop/source/
unrar x OXID_ESHOP_TESTS_EE_5.2.5_for_PHP_5.6_SOURCE.rar ./oxideshop/tests/
# Start the VM
vagrant up
[12] | The actual sharing process of the folder will be done by
the VM provision process, end-user only needs to create this folder and make
sure the folder exists. The path and name of the folder is configurable via
the vagrant_local.vm.app_shared_folder parameter. By default it's
<oxvm_eshop_root>/oxideshop . More information about how to configure this
value can be found in:
Change shared folder path . |
In order to provision only individual part of the VM one can simply use
ANSIBLE_TAGS
environment variable. Consider the following examples:
# Provision PHP and MySQL parts only
ANSIBLE_TAGS=php,mysql vagrant provision
# Provision OXID eShop related part only
ANSIBLE_TAGS=eshop vagrant provision
Ansible tags are marked inside roles
directories. To get a list of them try running the following command:
grep -r -A 2 --include="*.yml" "tags\:" .
List of troubleshooting items:
During the provision process (which could be invoked implicitly by
vagrant up
or explicitly by vagrant provision
) a task Run composer
install
might hang (waiting for time-out) because github access API limit
has been reached and composer
is asking for github account username/password
which could resolve the API limit. Ansible
will not provide this information
to STDOUT
or STDERR
so it will look like the task just hanged.
Since there are no options to provide username/password for this particular task one could just use a github API token which will allow to overcome the API access limit.
How to create and configure a github token is described in Define github token for composer chapter.
Older versions of eShop contains a very strict test inside Unit_Admin_ModuleListTest::testRender() method which tries to match the exact list of available modules. The test method might fail because VM includes SDK components and some of them are actual modules (which will result in modified list of available modules).
This is a known issue which is fixed in the development and new upcoming releases of eShop.
To check which shop is compatible with testing library please refer to compatibility list.
This message will only appear if a Zend Guard encoded eShop package is being used. In order to solve the issue one has to install Zend Guard Loader which will decode the encoded PHP files on execution.
To install and enable Zend Guard Loader PHP extension add the following configuration:
php:
zendguard:
install: true
Keep in mind that this extension will only work with the default PHP version, i.e. at the moment the use of extenion with phpbrew is not automated.
To install and enable Zend Guard Loader PHP extension manually inside the VM:
# From host (local machine)
vagrant ssh
# From guest (virtual machine)
cd /usr/lib/php/20131226/
sudo wget https://github.com/OXID-eSales/oxvm_assets/blob/master/zend-loader-php5.6-linux-x86_64.tar.gz?raw=true -O zend.tar.gz
sudo tar zxvf zend.tar.gz
sudo cp zend-loader-php5.6-linux-x86_64/ZendGuardLoader.so ./
sudo cp zend-loader-php5.6-linux-x86_64/opcache.so ./zend_opcache.so
cd /etc/php/5.6/mods-available/
sudo sh -c 'echo "zend_extension=ZendGuardLoader.so" > zend.ini'
sudo sh -c 'echo "zend_extension=zend_opcache.so" >> zend.ini'
sudo phpdismod opcache
sudo phpenmod zend
sudo service apache2 restart
Keep in mind that different PHP version needs different version of Zend Guard Loader extension. List of possible extension versions can be found in oxvm_assets repository.
More information on how to install and configure Zend Guard Loader can be found at: http://files.zend.com/help/Zend-Guard/content/installing_zend_guard_loader.htm
The example of error message:
{ oxvm_eshop } master » vagrant destroy
Vagrant is attempting to interface with the UI in a way that requires
a TTY. Most actions in Vagrant that require a TTY have configuration
switches to disable this requirement. Please do that or run Vagrant
with TTY.
Please check answers on stackoverflow for your specific case: http://stackoverflow.com/questions/23633276/vagrant-is-attempting-to-interface-with-the-ui-in-a-way-that-requires-a-tty
Restart selenium server is needed and can be done with command:
sudo /etc/init.d/selenium restart
In case there are Internet connection issues composer might take longer time to download
various packages and hit ProcessTimeOutException
. In order to avoid that configuration can
be updated to increase this time-out:
php:
composer:
timeout: 3000
Using user name with space in it leads to an error message:
The directory where plugins are installed (the Vagrant home directory) has a space in it. On Windows, there is a bug in Ruby when compiling plugins into directories with spaces. Please move your Vagrant home directory to a path without spaces and try again.
Possible solution:
- Install Vagrant in a directory which has no spaces in the path.
- Define Windows Environment Variable
%VAGRANT_HOME%
to hold path to the directoryPath_to_Vagrant\bin