Skip to content

Commit 5beb561

Browse files
stephengaitostephengaito
committed
initial commit of local ansible scripts to build test-build and test-install a pdf2htmlEX version
1 parent ed8de4d commit 5beb561

53 files changed

Lines changed: 961 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎building/.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
hostsEnv

‎building/Readme.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Ansible AWS
2+
3+
This Ansible project contains the information required to commission and
4+
setup a bare standard Ubuntu instance. This allows me to create a new bare
5+
standard instance in order to test compile packages on a know base system,
6+
and will help to explicilty determine what dependencies are required for a
7+
given project.
8+
9+
## Setup
10+
11+
Using the Ubuntu chooser and then the AWS console create a new EC2
12+
instance:
13+
14+
https://cloud-images.ubuntu.com/locator/ec2/
15+
16+
THEN edit the file `hostsEnv` to ensure the aws0, aws1, and/or aws2 are
17+
associated with the correct *public* ip address
18+
19+
THEN each time you change the `hostsEnv` file you need to type:
20+
21+
source ./hostsEnv
22+
23+
in each terminal you are using.
24+
25+
THEN ssh for the first time by typing one of the following:
26+
27+
./sshAWS0
28+
./sshAWS1
29+
./sshAWS2
30+
31+
This assumes you have an ssh key agent installed and working...
32+
33+
## Develop, test-building and test-installation
34+
35+
TO *develop* a new version of pdf2htmlEX taken from your *own* fork of the
36+
pdf2htmlEX repository type one of the following:
37+
38+
./runPlaybook0 ec2-install-pdf2html-develop.yml
39+
./runPlaybook1 ec2-install-pdf2html-develop.yml
40+
./runPlaybook2 ec2-install-pdf2html-develop.yml
41+
42+
TO *test* *build* an existing version of pdf2htmlEX from the *main*
43+
pdf2htmlEX repository type one of the following:
44+
45+
./runPlaybook0 ec2-install-pdf2html-master.yml
46+
./runPlaybook1 ec2-install-pdf2html-master.yml
47+
./runPlaybook2 ec2-install-pdf2html-master.yml
48+
49+
TO *test* *install* an existing *.deb package type one of the following:
50+
51+
./runPlaybook0 ec2-install-pdf2html-testDeb.yml
52+
./runPlaybook1 ec2-install-pdf2html-testDeb.yml
53+
./runPlaybook2 ec2-install-pdf2html-testDeb.yml
54+
55+
*In each case ensure you have the correct github repository tags specified
56+
in the above *.yml files*
57+
58+
## Testing pdf2htmlEX itself
59+
60+
To test pdf2htmlEX on the testing machine type one of the following:
61+
62+
./pdf2htmlTest testrevelatornl
63+
./pdf2htmlTest joyLoL
64+
65+
THEN open the resulting html in a web browser on you local machine by
66+
typing one of the following:
67+
68+
openHTML0
69+
openHTML1
70+
openHTML2
71+
72+
73+
## Teardown
74+
75+
Using the AWS console .... delete the unused instances
76+
77+
# pdf2htmlEX
78+
79+
## Build poppler
80+
81+
cd poppler
82+
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr \
83+
-DENABLE_LIBOPENJPEG=none
84+
make
85+
sudo make install
86+
87+
## Build pdf2htmlEX
88+
89+
cd pdf2htmlEX
90+
cmake .
91+
make
92+
sudo make install
93+

‎building/TipsAndTricks.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# Tips and Tricks
2+
3+
## Poppler ABI changes ;-(
4+
5+
Check all {inc,dec}RefCnt uses... from poppler-0.68.0 the USER is supposed
6+
to incRefCnt as well as decRefCnt.
7+
8+
## Finding hidden virtual function prototypes
9+
10+
### g++
11+
12+
make sure the -Woverloaded-virtual warnings are turned ON!
13+
14+
### clang++
15+
16+
make sure the -Woverloaded-virtual warnings are turned ON!
17+
18+
use clang-tidy:
19+
20+
clang-tidy -p build -header-filter=.* <<a compilation unit>>
21+
22+
This will highlight *clang-diagnostic-overloaded-virtual* warnings.
23+
24+
Consider adding the 'cert-*' checks to check for CERT Secure Coding
25+
Guidelines
26+
27+
## General tips
28+
29+
### g++
30+
31+
To get the list of g++ warnings:
32+
33+
g++ --help=warnings
34+
35+
### clang++
36+
37+
To get the list of clang++ warnings, see [Diagnostic flags in
38+
Clang](https://clang.llvm.org/docs/DiagnosticsReference.html).
39+
40+
Generally all warnings flags which g++ implements will be implemented by
41+
clang++. BUT they might be implemented differently (and hence triggered on
42+
slightly different conditions).
43+
44+
See: [Clang
45+
tools](https://github.com/llvm-mirror/clang/blob/master/docs/ClangTools.rst)
46+
47+
See:
48+
[Clang-tidy](https://github.com/llvm/llvm-project/blob/master/clang-tools-extra/docs/clang-tidy/index.rst)
49+
50+
See:
51+
[Clang-Tidy](https://clang.llvm.org/extra/clang-tidy/)
52+
53+
54+
See:
55+
[Clang-check](http://manpages.org/clang-check)
56+
57+
See:
58+
* [Clang-query]
59+
(https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-2-examining-the-clang-ast-with-clang-query/)
60+
* [AST Matcher
61+
Reference](https://clang.llvm.org/docs/LibASTMatchersReference.html)
62+
* [Matching the Clang AST](https://clang.llvm.org/docs/LibASTMatchers.html)

‎building/ToDo.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Things To Do
2+
3+
Add 16.04 setup
4+
5+
Instrament poppler GfxFont.cc getNextChar to help understand what is going
6+
wrong

‎building/ansible.cfg‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# This is the main Ansible configuration file for the vpnConfig
2+
3+
[defaults]
4+
inventory = hosts
5+
#roles_path = roles
6+
log_path = ./logs/ansible.log
7+
#callback_plugins = library/callback_plugins
8+

‎building/basicBuildSetup.yml‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
# This is the basic setup which is common across all of the various
3+
# platform scripts
4+
#
5+
# This "raw" playbook provisions an EC2 Instance for use as a temporary
6+
# trial compile machine
7+
#
8+
# See: https://www.agix.com.au/build-an-ec2-using-ansible-step-by-step/
9+
# and: https://www.linuxschoolonline.com/use-ansible-to-build-and-manage-aws-ec2-instances/
10+
11+
- name: Setup and install basic tools
12+
hosts: compileServers
13+
remote_user: ubuntu
14+
gather_facts: no
15+
pre_tasks:
16+
- name: 'install python'
17+
raw: 'sudo apt-get -y install python'
18+
become: yes
19+
tasks:
20+
- name: Update all existing packages
21+
become: yes
22+
apt:
23+
upgrade: yes
24+
update_cache: yes
25+
cache_valid_time: 3600 # one hour
26+
- name: Install Compile tools
27+
become: yes
28+
apt:
29+
state: latest
30+
update_cache: yes
31+
name:
32+
- aptitude
33+
- git
34+
- libpoppler-private-dev
35+
- pkg-config
36+
- ruby
37+
- cmake
38+
- make
39+
- gcc
40+
- g++
41+
- clang
42+
#- clang++ in clang package
43+
- clang-tidy
44+
#- clang-tools
45+
- libcairo-dev
46+
- libspiro-dev
47+
- libpng-dev
48+
- libjpeg-dev
49+
- libpoppler-dev
50+
- libpango1.0-dev
51+
- libfontforge-dev
52+
- poppler-data
53+
- openjdk-8-jre-headless # required for minimization of javascript
54+
#- openjdk-11-jre-headless # required for minimization of javascript
55+
- nginx # helps view the result pdf2html
56+
- tree # useful tool
57+
- devscripts # used for packagingTools
58+
- debhelper # used for packagingTools >= v0.18.0
59+
- python-dev # used for packagingTools
60+
- name: copy test scripts
61+
copy:
62+
src: scripts/
63+
dest: .
64+
mode: a+x
65+
- name: copy simple test PDF files
66+
copy:
67+
src: files/
68+
dest: .

‎building/basicTestSetup.yml‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
---
2+
# This is the basic setup which is common across all of the various
3+
# platform scripts
4+
#
5+
# This "raw" playbook provisions an EC2 Instance for use as a temporary
6+
# trial compile machine
7+
#
8+
# See: https://www.agix.com.au/build-an-ec2-using-ansible-step-by-step/
9+
# and: https://www.linuxschoolonline.com/use-ansible-to-build-and-manage-aws-ec2-instances/
10+
11+
- name: Setup and install basic tools
12+
hosts: compileServers
13+
remote_user: ubuntu
14+
gather_facts: no
15+
pre_tasks:
16+
- name: 'install python'
17+
raw: 'sudo apt-get -y install python'
18+
become: yes
19+
tasks:
20+
- name: Update all existing packages
21+
become: yes
22+
apt:
23+
upgrade: yes
24+
update_cache: yes
25+
cache_valid_time: 3600 # one hour
26+
- name: Install Test tools
27+
become: yes
28+
apt:
29+
state: latest
30+
update_cache: yes
31+
name:
32+
- aptitude
33+
- nginx
34+
- name: copy test scripts
35+
copy:
36+
src: scripts/
37+
dest: .
38+
mode: a+x
39+
- name: copy simple test PDF files
40+
copy:
41+
src: files/
42+
dest: .

‎building/ec2-install-pdf2html-develop.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
3+
# This "raw" playbook provisions an EC2 Instance for use as a temporary
4+
# trial compile machine
5+
#
6+
# See: https://www.agix.com.au/build-an-ec2-using-ansible-step-by-step/
7+
# and: https://www.linuxschoolonline.com/use-ansible-to-build-and-manage-aws-ec2-instances/
8+
9+
- import_playbook: basicBuildSetup.yml
10+
11+
- name: Setup and install poppler and pdf2htmlEX repos for compilation
12+
hosts: compileServers
13+
remote_user: ubuntu
14+
gather_facts: no
15+
tasks:
16+
- name: Install poppler git repo for compilation
17+
git:
18+
repo: https://gitlab.freedesktop.org/poppler/poppler.git
19+
dest: /home/ubuntu/poppler
20+
version: poppler-0.75.0
21+
- name: Install pdf2htmlEX git repo for compilation
22+
git:
23+
repo: https://github.com/stephengaito/pdf2htmlEX.git
24+
dest: /home/ubuntu/pdf2html
25+

‎building/ec2-install-pdf2html-master-tmp.yml‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
---
2+
3+
# This "raw" playbook provisions an EC2 Instance for use as a temporary
4+
# trial compile machine
5+
#
6+
# See: https://www.agix.com.au/build-an-ec2-using-ansible-step-by-step/
7+
# and: https://www.linuxschoolonline.com/use-ansible-to-build-and-manage-aws-ec2-instances/
8+
9+
- import_playbook: basicBuildSetup.yml
10+
11+
- name: Setup and install poppler and pdf2htmlEX repos for compilation
12+
hosts: compileServers
13+
remote_user: ubuntu
14+
gather_facts: no
15+
tasks:
16+
- name: Install poppler git repo for compilation
17+
git:
18+
repo: https://gitlab.freedesktop.org/poppler/poppler.git
19+
dest: /home/ubuntu/poppler
20+
version: poppler-0.74.0
21+
- name: Install pdf2htmlEX git repo for compilation
22+
git:
23+
repo: https://github.com/pdf2htmlEX/pdf2htmlEX.git
24+
dest: /home/ubuntu/pdf2html
25+
version: v0.18.0-poppler-0.74.0-ubuntu-19.04
26+
- name: Install pdf2htmlEX packaging tools
27+
git:
28+
repo: https://github.com/stephengaito/pdf2htmlEX-packagingTools
29+
dest: /home/ubuntu/pdf2html-packagingTools
30+
- name: copy build_dists.py
31+
copy:
32+
src: /home/ubuntu/pdf2html-packagingTools/build_dists.py
33+
remote_src: yes
34+
dest: /home/ubuntu/pdf2html/build_dists.py
35+
- name: copy debian directory
36+
copy:
37+
src: /home/ubuntu/pdf2html-packagingTools/debian
38+
remote_src: yes
39+
dest: /home/ubuntu/pdf2html
40+

‎building/ec2-install-pdf2html-master.yml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
3+
# This "raw" playbook provisions an EC2 Instance for use as a temporary
4+
# trial compile machine
5+
#
6+
# See: https://www.agix.com.au/build-an-ec2-using-ansible-step-by-step/
7+
# and: https://www.linuxschoolonline.com/use-ansible-to-build-and-manage-aws-ec2-instances/
8+
9+
- import_playbook: basicBuildSetup.yml
10+
11+
- name: Setup and install poppler and pdf2htmlEX repos for compilation
12+
hosts: compileServers
13+
remote_user: ubuntu
14+
gather_facts: no
15+
tasks:
16+
- name: Install poppler git repo for compilation
17+
git:
18+
repo: https://gitlab.freedesktop.org/poppler/poppler.git
19+
dest: /home/ubuntu/poppler
20+
version: poppler-0.74.0
21+
- name: Install pdf2htmlEX git repo for compilation
22+
git:
23+
repo: https://github.com/pdf2htmlEX/pdf2htmlEX.git
24+
dest: /home/ubuntu/pdf2html
25+

0 commit comments

Comments
 (0)