Labels

Linux (6) OpenCV (4) Deep Learning (3) MATLAB (3) Mac OS X (3) Windows (2) C# (1) Node JS (1)

2017年9月21日 星期四

Use Deep Learning Tool inside Unreal

In this post I would like to introduce how to setup Unreal as a deep reinforcement learning environment. Before going on please get your deep learning environment ready and install TensorFlow or Theano. Our environment is based on the amazing Unreal Python Engine created by 20tab:
https://github.com/20tab/UnrealEnginePython

I've successfully installed on Ubuntu, Windows and Mac. Installing on Windows & Mac are relatively easy so let's start with Ubuntu first.

Ubuntu

1. Installing Unreal

First you need to register an account on EpicGame website. For linux we need to compile from source code.  Make sure you get granted for downloading from github then execute the commands below:

# Get authorized on github by EpicGames first. 
# Can use -b to specify UE4 version. Ex: git clone -b 4.16
git clone https://github.com/EpicGames/UnrealEngine.git
cd UnrealEngine
./Setup.sh
./GenerateProjectFiles.sh
make
#....wait for 30+ mins....
# Run UE4 editor
./Engine/Binaries/Linux/UE4Editor

1.1 Unreal Terminology

There are some must-know terminologies of Unreal:

LEVEL: A gaming region, saved as (.umap) so it's also called Map ACTOR: Any movable object in a level
PAWN: Subclass of Actor controlled by player or AI (NPC) BLUEPRINT: Visual game scripting system

More terminologies can be found here.

1.2 Learning Unreal Basics

To create your own test environment you need to learn how to manipulate objects (actors) and write scripts. There are lots of tutorials on Unreal website and I suggest to read the following first:

a. Level Designer Quick Start



b. Unreal Engine 4 Tutorial for Beginners: Getting Started


c. Blueprints Quick Start Guide


NOTE: Unfortunately there is no Epic Games launcher for Linux, so we need to download free examples using Windows or Mac and then upload to Linux.

2. Installing UnrealPythonEngine

Now we need to install the UnrealPythonEngine plugin from
https://github.com/20tab/UnrealEnginePython
By default Unreal will load and compile any plugs in /Plugins folder under the project directory. Note we need to create C++ project, not Blueprint:



The installation steps are listed as below:
  • Create a new Unreal C++ project and close the editor once the project is fully started
  • Go to the just created project directory and create the Plugins folder
  • move to the Plugins folder and clone the UnrealPythonEngine plugin:
  • git clone https://github.com/20tab/UnrealEnginePython
Before we re-open Unreal project and compile the plugin, we need to setup Python engine first.

2.1 Choose Python 2.7 or Python 3.5+

The default Python engine is 3.6. To change it we need to edit the build file:
Plugins/UnrealEnginePython/Source/UnrealEnginePython/UnrealEnginePython.Build.cs

And add your Python installation folder into pythonHome as below:
public class UnrealEnginePython : ModuleRules
{ 
  
    // leave this string as empty for triggering auto-discovery of python installations...
    private string pythonHome = "/usr/include/python2.7;/usr/lib/x86_64-linux-gnu/libpython2.7.so";
    ......

Reopen the project and start compiling the plugin, a "not compatible" warning message may be shown:

Ignore the warning and click "No", and you will see the next dialog

Click "Yes"

*Compiling error for UnrealPythonEngine

If you are using clang++-3.9 and older Unreal version (like 4.15) to compile UnrealPythonEngine, you will encounter the following error:

ThirdParty/FBX/2016.1.1/include/fbxsdk/core/fbxproperty.h:1242:70: error: binding dereferenced null pointer to reference has undefined behavior [-Werror,-Wnull-dereference] return StaticInit(pObject, pName, FbxGetDataTypeFromEnum(FbxTypeOf(*((FbxReference*)0))), pValue, pForceSet, pFlags); 

To fix it, change your clang++ version to 3.8 or lower:
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.8 100
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.8 100


2.2 Open Python Console

Now we are ready to run some Python scripts. Make sure your deep learning tool (TensorFlow or Theano) is accessiable for the UE4Editor. If VirtualEnv is used, activate the environment before running UE4, for example:
/data/UnrealEngine-4.17$ source ~/theano/bin/activate
(theano) /data/UnrealEngine-4.17$ ./Engine/Binaries/Linux/UE4Editor

Open the Unreal project and select Window -> Developer Tool -> Python Console

Import your Deep Learning framework, then you are ready to go!

Windows & Mac

For Windows and Mac, just download the binary releases from Github of UnrealPythonEngine and unzip it in your project's Plugins folder. Make sure you download the right Python version.

2017年9月6日 星期三

Top Conferences in Data Mining, Computer Vision and Artificial Intelligence

Note: The conferences are sorted by deadline.

Data Mining

Short Name
Full Name
Deadline
When
KDD
Knowledge Discovery and Data Mining
February
August
ASONAM
Advances in Social Networks Analysis and Mining
March
July/Aug.
ECML + PKDD
The European Conference on Machine Learning & Principles and Practice of Knowledge Discovery in Databases
April
September
CIKM
IEEE International Conference on Information and Knowledge Management
May
November
ICDM
IEEE International Conference on Data Mining
June
November
WSDM
Web Search and Data Mining
August
February
ICDE
International Conference on Data Engineering
Sep./Oct.
April
SDM
SIAM International Conference on Data Mining
October
May
PAKDD
The Pacific-Asia Conference on Knowledge Discovery and Data Mining
Oct./Nov.
May

Computer Vision & Image Processing

Short Name
Full Name
Deadline
When
ICIP
International Conference on Image Processing
January
September
SIGGRAPH
Computer Graphics and Interactive Techniques
January
August
ICPR
International Conference on Pattern Recognition #
January
August
ECCV
European Conference on Computer Vision#
February
September
ICCV
International Conference on Computer Vision*
March
October
ACM MM
ACM International Conference on Multimedia
April
October
CVPR
Computer Vision and Pattern Recognition
November
June/July
ICME
International Conference on Multimedia and Expo
December
July
# Every even year  * Every odd year

Machine Learning & Artificial Intelligence

Short Name
Full Name
Deadline
When
IJCAI
International Joint Conference on Artificial Intelligence
February
August
ICML
International Conference on Machine Learning
February
August
NIPS
Neural Information Processing Systems Conference
May
December
AAAI
American Association for AI National Conference
September
February
ICLR
International Conference on Leanring Representations
October
April/May

Robotics

Short Name
Full Name
Deadline
When
IROS
International Conference on Intelligent Robots & Systems
March
September
ICRA
International Conference on Robotics and Automation
September
May