Deployment, UI compiling twice, token error and eslint

#1

Dear community and kobo developers!

Pardon me if my questions are still quite noobish … the learning curve for this stack is steep.

I would like to ask if anyone has had a detailed look at the outputs during a local deployment, when building (not pulling) kpi.

My understanding is that npm run build and python manage.py collecstatic --noinput are run a first time in the Dockerfile. BUT … I cannot understand how and why it is run again when I do docker-compose up!

I did check the /kpi/docker/init.bash and can see:

if [[ ! -d "${KPI_SRC_DIR}/staticfiles" ]] || ! python "${KPI_SRC_DIR}/docker/check_kpi_prefix_outdated.py"; then
    echo 'Building static files from live code.'
    (cd "${KPI_SRC_DIR}" && npm run build && python manage.py collectstatic --noinput)
fi

KPI_PREFIX is set to be KPI_PREFIX=/ in the env_file: ./envfiles/kpi.txt. Thus, when running docker-compose up everything should be fine! And the variable available in the environment. But the python script check_kpi_prefix_outdated.py always fails because of a key error ['KPI_PREFIX']. hence … it compiles everything again at deployment. Meanwhile, the other containers are waiting for kpi to be up …

Last but not least, I cannot understand the following:

  1. If uncomment multiple lines in /kpi/jsapp/js/components/header.es6 using {/* */} (as I understand it should be done in .es6 files.
  2. Then npm run build during the build of kpi works fine!
  3. But, when docker-compose up … it tries to compile again (see aforementioned issue) and suddenly fails with a token error pointing at the ending } of my uncommented lines and sends me an error referring to /srv/src/kpi/node_modules/eslint-loader/index.js.

If anyone could help, it would be much appreciated :slight_smile: cause I am pulling my hair at the moment.

Best,

Michel-Pierre

#2

Hi Michel-Pierre,

I don’t have the trouble with check_kpi_prefix_outdated.py that you do. The KeyError would indicate that KPI_PREFIX is missing from the environment, no? Here’s what I see from within my kpi container:

root@kpi:/srv/src/kpi# echo $KPI_PREFIX
/
root@kpi:/srv/src/kpi# python docker/check_kpi_prefix_outdated.py && echo 'Great!'
Great!

Regarding changes to header.es6, it’s worth noting the distinction between ES6 and JSX. You’ll find JSX (it resembles HTML) within KoBo’s ES6 files, due to the use of React, and inside JSX you do need to surround comments with braces. However, ES6 itself uses the familiar /* */ and // comments.

It’s odd that the npm run build during the Docker build works but docker-compose up does not. If eslint is to blame, you can try disabling it by commenting out or removing these lines in the webpack configuration:

#3

Hi jnm and thanks a lot for your response!

Thanks for the clarification on ES6 and JSX, now it works.

I have not managed to sort out the double compile though …

So no matter what I do I keep getting the same double build, one when docker-compose build kpi and one when docker-compose up. I cannot figure out why and this is what I end up with in my kpi container:

root@kpi:/srv/static/compiled#
-rw-r--r--  1 www-data root 4271122 Mar 26 11:47 app-d257e8cc804e5fd3d173.js
-rw-r--r--  1 www-data root 4270367 Mar 26 11:47 app-f6b162c1d88e99745df5.js

What is also very odd, is that I end up with a mess of webpack-stats.json files.
kpi:/srv/src/kpi/webpack-stats.json : points to the most recent version of the app (compiled when docker-compose up)
kpi:/srv/webpack-stats.json : points to the inital version, compiled when docker-compose build.
AND,

/srv/src/kpi/webpack-stats.json should be a symlink according to the kpi Dockerfile:

ln -s "${WEBPACK_STATS_PATH}" webpack-stats.json

… but it is not :frowning:

and once up and running, I have the same output on my kpi container:

root@kpi:/srv/src/kpi# echo $KPI_PREFIX
/
root@kpi:/srv/src/kpi# python docker/check_kpi_prefix_outdated.py && echo 'Great!' Great!

I wonder if that is not also leading to further issues down the line cause when I try to reload or hot reload anything, my browser keep loading an older version of the app that is not even in webpack-stats.json. I looked at everything, such as WEBPACK_LOADER and {% render_bundle 'app' 'js' %} but for some reason it does not pick the version in /srv/src/kpi/webpack-stats.json. It sometimes load the first version that was loaded at docker-compose up … sometimes picks a version from devServer … it is completely random and mysterious :confused: :face_with_monocle:

Any help would be much appreciated :slight_smile: and thanks again

#4

Also, I actually meant to ask

How do you track/debug the app-xxxx.js version that is being loaded? I sometimes get completely random hashes being loaded according to the developer tool of google. And when I try to find these hashes, I cannot find them anywhere. Are they somewhere in the the memory … ? of django, of the browser, is it a react thing?

Apologies again if these questions are basic, I am still quite new to the development of such a stack and trying to figure how to get a proper flow for debugging and contributing to the toolbox.

#5

I had similar problems that you describe. I found this in an old thread:

sometimes I have this problem that after working some time, I get some old/cached js file (changes in the code no longer reflect in the browser) and if I re-run npm watch script I get 404

My investigation lead me to django-webpack-loader and {% render_bundle 'app' 'js' %} as you did.

I temporarily was replacing this line with a <script> with url that I’ve got from my local kpi/webpack-stats.json file. This was ok for short time :wink:

Things I’ve tried:

  1. in kobo-docker/envfiles/nginx.txt set KPI_WEB_SERVER=runserver_plus and then restart the container
  2. changing WEBPACK_LOADER configuration in settings.py:
    WEBPACK_LOADER = {
        'DEFAULT': {
            'CACHE': not DEBUG,
            'STATS_FILE': os.path.join(BASE_DIR, 'webpack-stats.json'),
    
  3. commenting out lines 70-71 from kpi/Dockerfile:
    # FIXME: Move `webpack-stats.json` to some build target directory so these ad-hoc workarounds don't continue to accumulate.
    ln -s "${WEBPACK_STATS_PATH}" webpack-stats.json
    

I don’t remember if any of these really helped. After I switched my development workflow to https://github.com/kobotoolbox/kobo-install I don’t remember having this problem ever again.