1Building and Running on Windows 2=============================== 3Note: See also [BUILDING on Windows](/README.md#building-on-windows) 4in the main README. 5 6This directory contains project files for building Link Grammar with the 7Microsoft Visual Studio 2019 IDE (MSVC 16). They were created and tested with 8the Community Edition of that product. 9 10**!!!WARNING!!!**<br> 11In the examples below, "console-prompt>" at start of line means the console 12prompt. Don't type it because the result could be destructive! 13 14Supported target versions 15------------------------- 16The intention is to support versions from Vista on (some WIN32 17functions which are used are not supported in earlier versions.) 18 19The system compatibility definitions: 20In each project file - Target Platform version: 8.1 21In the `MSVC-common` property sheet - **Common properties->User Macros/CFLAGS**: 22 23`NTDDI_VERSION=NTDDI_VISTA;_WIN32_WINNT=_WIN32_WINNT_VISTA;` 24 25Dependencies 26------------ 27The regex package, which includes libraries and header files, must be 28separately downloaded. Also see GNUREGEX_DIR below. 29 30For Python bindings, install the desired Python distributions from 31[Python Releases for Windows](https://www.python.org/downloads/windows/). 32You also have to install [SWIG](http://www.swig.org/download.html). 33 34The bindings were testes using swigwin-3.0.10 with Python 3.4.4. 35 36Setup 37----- 38The build files make use of User Macros, defined in the property 39sheet "Local", as follows: 40 41- GNUREGEX_DIR must be pointing to an unzipped POSIX regex distribution 42 (which has the subdirectories `include` and `lib`). Its default is 43 `%HOMEDRIVE%%HOMEPATH%\Libraries\gnuregex`. 44 If the environment variable GNUREGEX exists, its value is used instead. 45 The library file name is assumed to be "regex.lib" and needs to be 46 generated from the DLL if it is not included in the distribution (see 47 http://stackoverflow.com/questions/9946322). 48 49 A tested library can be downloaded from 50 [Tre for Windows](http://gnuwin32.sourceforge.net/packages/tre.htm). 51 If your system is 64-bits, use the provided 64-bits library. 52 53 The corresponding lib file is missing there. If you have Cygwin 54 installed, you can generate it as follows (console command): 55 56``` 57dlltool -l regex.lib -d libtre/win32/tre.def -D regex.dll libtre/win32/bin/x64_release/libtre_dll.dll 58``` 59 60- LG_DLLPATH should include the directory of the regex DLL, which is 61 normally under GNUREGEX_DIR (or it can be a central DLL directory etc.). 62 The default is `$(GNUREGEX_DIR)\lib`. 63 64- JAVA_HOME, if used, must be pointing to a locally installed SDK/JDK, 65 which has the subdirectories "include" and `include/win32` (defined in 66 the LinkGrammarJava project under **Common properties->C/C++->General-> 67 Additional Include Directories**. 68 If your JAVA SDK/JDK installation has defined the JAVA_HOME environment 69 variable (check it) then there is no need to define this User Macro. 70 71- WINFLEXBISON should be the directory of the "Win flex-bison" project, 72 as downloaded from its [Web site](https://winflexbison.sourceforge.io/). 73 Tested with version 2.5.9. 74 Leave it blank if would like to use a ready **pp_lexer.c** file. 75 The default is **C:\win_flex_bison**. 76 77### Definitions for Python bindings 78 79 Macro | Default value | 80---|---| 81PYTHON3 | C:\Python34 | 82PYTHON3_INCLUDE | $(PYTHON3)\include | 83PYTHON3_LIB | $(PYTHON3)\lib | 84PYTHON3_EXE | $(PYTHON3)\python.exe | 85 86If you want to build any of the bindings, make sure it is marked for build 87in the configuration manager, or select "build" for the desired bindings in 88Solution Explorer. 89 90Compiling 91--------- 92- Compiling and running got checked on Windows 10, but is intended to be 93 compatible to Vista and on (XP is not supported). 94 95- To compile LinkGrammar, open the solution file LinkGrammar.sln, change 96 the solution platform to x64 if desired, and build. 97 98- The solution configuration is configured to create a debug version by 99 default. It's probably a good idea to switch a "Release" configuration. 100 You can do this at **Build Menu->Configuration Manager**. 101 102- The wordgraph-display feature is enabled when compiled with 103 USE_WORDGRAPH_DISPLAY (already defined in the `LGlib-features` property sheet 104 - **Common properties->C/User Macros/DEFS**). 105 106- By default, the library is configured to create a DLL. If you want 107 to instead build a static library, the macro LINK_GRAMMAR_STATIC must 108 be defined before the inclusion of any header files for both the compiling 109 of the link-grammar library and for the application that uses it. Other 110 compiler settings will also have to be changed to create a static library 111 of course. 112 113Running 114------- 115The last step of the "LinkGrammarExe" project creates `link-parser.bat`. Copy 116it to somewhere in your PATH and then customize it if needed (don't customize 117it in-place, since rebuilding would overwrite it). Note that it prepends PATH 118with the User Macro LG_DLLPATH (see Setup above). If you would like to display 119the wordgraph (see below) set also the PATH for `dot.exe` if it is not already 120in your PATH. 121 122If USE_WORDGRAPH_DISPLAY has been used when compiling (the default), then 123typing `!test=w`g at the `linkparser>` prompt can be used in order to display 124the wordgraph of the next sentences to be parsed. See 125[Word-graph display](/linkgrammar/tokenize/README.md#word-graph-display) 126in `/linkgrammar/tokenize/README.md`. 127for how to use optional display flags. By default, `PhotoViewer.dll` is 128invoked to display the graph. If X11 is available and your `dot.exe` command 129has the "xlib" driver, it can be used to display the wordgraph when the x flag 130is set (i.e. `!test=wg:x`), for example when running under Cygwin/X. In any 131case it uses the `dot` command of Graphviz. Graphviz can be installed as part 132of Cygwin (in which case it included the "xlib" driver"), or separately from 133[Graphviz](http://www.graphviz.org/Download_windows.php). 134Both `PhotoViewer.dll` (if used) and `dot` must be in your PATH (if needed, 135you can customized that in a copy of `link-parser.bat`, as described above). 136 137BTW, when running an MSVC-compiled binary under Cygwin, don't exit 138link-parser by using `^Z` - the shell may get stuck because the program 139somehow may continue to run in the background. Instead, exit using `!exit` . 140 141NOTE: The created DLLs need the MSVC 16 runtime environment to run. This is 142normally already installed in your machine with the installation of the IDE. 143But to be able to run Link Grammar on other computer you need to install 144[Visual C++ Redistributable for Visual Studio 2019](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads). 145This redistributable does not contain debug version of the MSVC runtime, so 146only "Release" Link Grammar will work with it. 147 148Running Python programs 149----------------------- 150Since the Link Grammar library has no installation script yet, 151running Python programs that use the bindings needs a careful setup 152of PYTHONPATH and PATH. A program named `make-check.py` (named after 153"make check" that runs `tests.py` in POSIX systems) is provided to 154set them automatically. 155 156Also see "Permanent installation". 157 158### Using make-check.py 159The `make-check.py` program is designed to reside in the MSVC 160configuration directory. However, it can run from any directory 161using a full or a relative path to invoke it. 162 163Usage: 164``` 165console-prompt>make-check [PYTHON_FLAG] PYTHON_OUTDIR [script.py] [ARGUMENTS] 166``` 167 168- PYTHON_FLAG: Optional flag for the Python program, e.g. `-vv` to debug 169imports. 170- PYTHON_OUTDIR: The directory to which the Python bindings got written.<br> 171For example, `x64\Release\Python3`. 172- script.py: Path leading to the script. If only a filename is specified 173(i.e. no `\` in the path) the specified script file is taken from the 174`bindings\python-examples\` directory. In order to run tests.py in 175its default location you can leave it empty. 176- ARGUMENTS: Optional script arguments, for example `-v` for `tests.py`. 177 178So in order to run `tests.py` with Python3 for a Debug compilation on x64 179platform, enter: 180``` 181console-prompt>make-check x64\Debug\Python3 182``` 183To debug a Python3 script "mylgtest.py" that resides in 184`bindings\python-examples`: 185``` 186console-prompt>make-check -mpdb x64\Debug\Python3 mylgtest.py 187``` 188To run a script in your home directory: 189``` 190console-prompt>make-check x64\Debug\Python3 \Users\username\mylgtest.py 191``` 192The following starts an interactive Python with the correct PYTHONPATH 193and PATH: 194``` 195console-prompt>make-check.py x64\Debug\Python3 "" 196``` 197Locale and code page settings 198----------------------------- 199In this version, the language dictionaries under the data directory define 200the locale that they need in order to process input sentences and the 201library automatically uses this locale, in a manner that is intended to be 202compatible with creating several different dictionaries per thread. 203 204If you use a dictionary which doesn't have this definition, or would like 205to set the default language when link-parser is invoked without a language 206argument, you can set the locale using the environment variable LANG, 207using a 2-letter language code and a 2-letter country code: 208``` 209console-prompt>set LANG=ll-CC 210``` 211For example: 212``` 213console-prompt>set LANG=en-US 214``` 215 216If you open a dictionary which doesn't have a locale definition, from a 217program (C program or a language with LG bindings) you have to set the 218correct locale (for the whole program or the particular thread)for the 219dictionary before creating the dictionary. Alternatively, the LANG 220environment variable can be set before creating the dictionary. 221 222The code page of the console should not be changed, unless it is desired 223to pipe the output to another program. In that case, this other program 224may read garbage due to a limitation in the way cmd.exe implements pipes. 225This can be solved as following ("more" is the example program) 226``` 227link-parser | (chcp 65001 & more) 228``` 229In that case `more` will be able to read UTF-8 input. However, the 230display may still not be perfect due to additional cmd.exe limitations. 231The code page can be changed manually to 65001, see below in the "Note for 232Python bindings". 233 234### A note for the Python bindings 235In order to produce UTF-8 output to the console, it needs use the CP_UTF8 236code page (65001): 237 238``` 239console-prompt>chcp 240Active code page NNNN 241console-prompt>chcp 65001 242Active code page: 65001 243``` 244 245Other programs may malfunction with this code page, so it can be restored 246when needed (or this console window can be used only for working with 247link-grammar). The link-parser command also changes the code page to 248CP_UTF8, but it restores the original one on exit. 249 250Console fonts 251------------- 252Courier New may be appropriate for all the languages in the data 253directory. If you don't have it in the console menu, it can be added 254through the registry (Google it). 255 256Permanent installation 257---------------------- 258For using the library independently of the build directory: 259 2601) If Python bindings were generated, copy the following modules to a 261 directory `linkgrammar` in a fixed location: `linkgrammar.py`, 262 `clinkgrammar.py`, `__init__.py`, `_clinkgrammar.pyd`. 263 264 Set the PYTHONPATH environment variable to point to the said 265 "linkgrammar" directory. 266 2672) Copy the link-grammar DLL to a fixed location.<br> 268 Add the DLL location permanently to PATH. 269 2703) Copy the `data` directory to the location of the DLL so it will get found. 271 272 273Implementation notes: 274--------------------- 275 276- The file `link-grammar/link-features.h.in` has a Custom Build Tool definition 277 which invokes `mk-link-features-h` to generate 278 `link-grammar/link-features.h`, with LINK_*_VERSION variable replacement as 279 defined in `configure.ac`. 280 281- The project file `LinkGrammarExe` has a Post-Build-Event definition for 282 generating `link-parser.bat`. 283 284 285Using a remote network share 286---------------------------- 287In order to use a link-grammar source repository from a network share (e.g. via 288Samba), and still be able to use the custom build steps in the Project files, 289there is a need to "convince" Windows it is a local filesystem. Else you will 290get "UNC path are not supported." on the batch runs, with bad results. This 291method will also allow the `link-parser.bat` file to run. (For other solutions 292see https://stackoverflow.com/questions/9013941). You will need to find out by 293yourself if this makes a security or another problem in your case. 294 295Here is what worked for me:<br> 296Suppose you use `host:/usr/local/src` remotely as share `src`: 297``` 298mklink /J src-j \\host\src 299mklink /D src src-link 300``` 301The second one needs administrator privileges.<br> 302Then use the repository through `src`. 303 304Debugging hints 305-------------- 306If, when starting the program under the debugger (by "Local Windows Debugger", 307"Debug->Start Debugging" (F5), etc.), `regex.dll` is not found, it can be 308added to the search `PATH` as follows:<br> 309Enter to LinkGrammarExe's Property Pages:<br> 310 `Solution Explorer->LinkGrammarExe->Properties`<br> 311Click on the writable location of:<br> 312 `Debugging->Environment`<br> 313Put there (LG_DLLPATH is defined in the `Local` Property pages):<br> 314`PATH=$(LG_DLLPATH)`<br> 315Make sure "Merge Environment" there is `Yes`. 316 317(The result is kept in `.user` Property Pages that are not part of the 318LG repository). 319