There is at least two ways to override template file and customize the html wich is rendered by Pushword : First is App Way (app per app), second is the bundle way (global).
Simplest way is to override it (partially or completly) by create a new file in ./templates/{$host}/page naming it like the default one
Eg: Overriding the default navbar can be done creating a file ./templates/{$host}/page/_navbar.html.twig.
You can see how it's handle for the documentation.
Every package (even core) is built like a symfony bundle, so you can override template file the bundle way.
For this, you just need to find the template file you want to override and create a copy inside your templates/bundles folder like.
Example, to override /page/page_default.html.twig :
./templates/bundles/PushwordCoreBundle/page/page_default.html.twig