Blame doc/development.rst

Pierre-Yves Chibon 300c54
Development
Pierre-Yves Chibon 300c54
===========
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Get the sources
Pierre-Yves Chibon 300c54
---------------
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Anonymous:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 7013cc
  git clone https://pagure.io/pagure.git
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Contributors:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
Pradeep CE (cep) 19e248
  git clone ssh://git@pagure.io/pagure.git
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Dependencies
Pierre-Yves Chibon 300c54
------------
Pierre-Yves Chibon 300c54
bill auger 74c1fa
Install the build dependencies of pagure:
bill auger 74c1fa
bill auger 74c1fa
::
bill auger 74c1fa
bill auger 74c1fa
  sudo dnf install git python-virtualenv libgit2-devel \
bill auger 74c1fa
                   libjpeg-devel gcc libffi-devel redhat-rpm-config
bill auger 74c1fa
bill auger 74c1fa
bill auger 74c1fa
The python dependencies of pagure are listed in the file ``requirements.txt``
Pierre-Yves Chibon 300c54
at the top level of the sources.
Pierre-Yves Chibon 300c54
bill auger 74c1fa
::
bill auger 74c1fa
bill auger 74c1fa
  virtualenv pagure_env
bill auger 74c1fa
  source ./pagure_env/bin/activate
bill auger 74c1fa
  pip install pygit2==<version found="" libgit2="" of="">.* # e.g. 0.23.*</version>
bill auger 74c1fa
  pip install -r requirements.txt
bill auger 74c1fa
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
.. note:: working in a `virtualenv <http: en="" latest="" www.virtualenv.org="">`_</http:>
Pierre-Yves Chibon 300c54
          is tricky due to the dependency on `pygit2 <http: www.pygit2.org="">`_</http:>
Pierre-Yves Chibon 300c54
          and thus on `libgit2 <https: libgit2.github.com="">`_</https:>
Pierre-Yves Chibon 300c54
          but the pygit2 `documentation has a solution for this
Pierre-Yves Chibon 300c54
          <http: install.html#libgit2-within-a-virtual-environment="" www.pygit2.org="">`_.</http:>
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Run pagure for development
Pierre-Yves Chibon d5796d
--------------------------
Pierre-Yves Chibon 300c54
Adjust the configuration file (secret key, database URL, admin group...)
Pierre-Yves Chibon 300c54
See :doc:`configuration` for more detailed information about the
Pierre-Yves Chibon 300c54
configuration.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Create the database scheme::
Pierre-Yves Chibon 300c54
bill auger 74c1fa
  ./createdb.py
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Create the folder that will receive the different git repositories:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 4e55c5
    mkdir {repos,docs,forks,tickets,requests,remotes}
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Run the server:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
bill auger 74c1fa
    ./runserver.py
Pierre-Yves Chibon 7013cc
Pierre-Yves Chibon 8ca915
If you want to change some configuration key you can create a file, place
Pierre-Yves Chibon 8ca915
the configuration change in it and use it with
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
::
Pierre-Yves Chibon 8ca915
bill auger 74c1fa
    ./runserver.py -c <config_file></config_file>
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
For example, create the file ``config`` with in it:
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
::
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
    from datetime import timedelta
Pierre-Yves Chibon 8ca915
    # Makes the admin session longer
Pierre-Yves Chibon 8ca915
    ADMIN_SESSION_LIFETIME = timedelta(minutes=20000000)
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
    # Use a postgresql database instead of sqlite
Pierre-Yves Chibon 8ca915
    DB_URL = 'postgresql://user:pass@localhost/pagure'
Pierre-Yves Chibon 8ca915
    # Change the OpenID endpoint
Pierre-Yves Chibon 8ca915
    FAS_OPENID_ENDPOINT = 'https://id.stg.fedoraproject.org'
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
    APP_URL = '*'
Pierre-Yves Chibon 8ca915
    EVENTSOURCE_SOURCE = 'http://localhost:8080'
Pierre-Yves Chibon 8ca915
    EVENTSOURCE_PORT = '8080'
Pierre-Yves Chibon 8ca915
    DOC_APP_URL = '*'
Pierre-Yves Chibon 8ca915
René Genz 520020
    # Avoid sending email when developing
Pierre-Yves Chibon 8ca915
    EMAIL_SEND = False
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
and run the server with:
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 8ca915
::
Pierre-Yves Chibon 8ca915
bill auger 74c1fa
    ./runserver.py -c config
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 300c54
To get some profiling information you can also run it as:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 8ca915
Pierre-Yves Chibon 300c54
    ./runserver.py --profile
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
You should be able to access the server at http://localhost:5000
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Every time you save a file, the project will be automatically restarted
Sergio Durigan Junior 7fae95
so you can see your change immediately.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon f87eeb
Create a pull-request for testing
Pierre-Yves Chibon f87eeb
----------------------------------
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
When working on pagure, it is pretty often that one wanted to work on a
Pierre-Yves Chibon f87eeb
feature or a bug related to pull-requests needs to create one.
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
Making a pull-request for development purposes isn't hard, if you remember
Pierre-Yves Chibon f87eeb
that since you're running a local instance, the git repos created in your
Pierre-Yves Chibon f87eeb
pagure instance are also local.
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon c6fea5
So here are in a few steps that one could perform to create a pull-request in a
Pierre-Yves Chibon f87eeb
local pagure instance.
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
* Create a project on your pagure instance, let's say it will be called ``test``
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon c6fea5
* Create a folder ``clones`` somewhere in your system (you probably do not
Pierre-Yves Chibon c6fea5
  want it in the ``repos`` folder created above, next to it is fine though)::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
    mkdir clones
Pierre-Yves Chibon f87eeb
rahul Bajaj 79c23b
* Clone the repo of the ``test`` project into this ``clones`` folder and move into it::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
    cd clones
Pierre-Yves Chibon f87eeb
    git clone ~/path/to/pagure/repos/test.git
rahul Bajaj 79c23b
    cd test
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
* Add and commit some files::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
    echo "*~" > .gitignore
Pierre-Yves Chibon f87eeb
    git add .gitignore
Pierre-Yves Chibon f87eeb
    git commit -m "Add a .gitignore file"
Pierre-Yves Chibon f87eeb
    echo "BSD" > LICENSE
Pierre-Yves Chibon f87eeb
    git add LICENSE
Pierre-Yves Chibon f87eeb
    git commit -m "Add a LICENSE file"
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
* Push these changes::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon c6fea5
    git push -u origin master
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
* Create a new branch and add a commit in it::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
    git branch new_branch
Pierre-Yves Chibon f87eeb
    git checkout new_branch
Pierre-Yves Chibon f87eeb
    touch test
Pierre-Yves Chibon f87eeb
    git add test
Pierre-Yves Chibon f87eeb
    git commit -m "Add file: test"
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
* Push this new branch::
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon c6fea5
    git push -u origin new_branch
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
Then go back to your pagure instance running in your web-browser, check the
Pierre-Yves Chibon c6fea5
``test`` project. You should see two branches: ``master`` and ``new_branch``
Pierre-Yves Chibon f87eeb
from there you should be able to open a new pull-request, either from the
Pierre-Yves Chibon f87eeb
front page or via the ``File Pull Request`` button in the ``Pull Requests``
Pierre-Yves Chibon f87eeb
page.
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon f87eeb
Pierre-Yves Chibon 300c54
Coding standards
Pierre-Yves Chibon 300c54
----------------
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
We are trying to make the code `PEP8-compliant
Pierre-Yves Chibon 300c54
<http: dev="" pep-0008="" peps="" www.python.org="">`_.  There is a `pep8 tool</http:>
Pierre-Yves Chibon 300c54
<http: pep8="" pypi="" pypi.python.org="">`_ that can automatically check</http:>
Pierre-Yves Chibon 300c54
your source.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
We are also inspecting the code using `pylint
Pierre-Yves Chibon 300c54
<http: pylint="" pypi="" pypi.python.org="">`_ and aim of course for a 10/10 code</http:>
René Genz 520020
(but it is an asymptotic goal).
Pierre-Yves Chibon 300c54
rahul Bajaj af6966
.. note:: both pep8 and pylint are available in Fedora:
rahul Bajaj af6966
rahul Bajaj af6966
          ::
rahul Bajaj af6966
rahul Bajaj af6966
            dnf install python-pep8 pylint
rahul Bajaj af6966
rahul Bajaj af6966
          or
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
          ::
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
            yum install python-pep8 pylint
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Send patch
Pierre-Yves Chibon 300c54
----------
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
The easiest way to work on pagure is to make your own branch in git, make
Pierre-Yves Chibon 300c54
your changes to this branch, commit whenever you want, rebase on master,
Pierre-Yves Chibon 300c54
whenever you need and when you are done, send the patch either by email,
Pierre-Yves Chibon 300c54
via the trac or a pull-request (using git or github).
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
The workflow would therefore be something like:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
   git branch <my_shiny_feature></my_shiny_feature>
Pierre-Yves Chibon 300c54
   git checkout <my_shiny_feature></my_shiny_feature>
Pierre-Yves Chibon 300c54
   <work></work>
Pierre-Yves Chibon 300c54
   git commit file1 file2
Pierre-Yves Chibon 300c54
   <more work=""></more>
Pierre-Yves Chibon 300c54
   git commit file3 file4
Pierre-Yves Chibon 300c54
   git checkout master
Pierre-Yves Chibon 300c54
   git pull
Pierre-Yves Chibon 300c54
   git checkout <my_shiny_feature></my_shiny_feature>
Pierre-Yves Chibon 300c54
   git rebase master
Pierre-Yves Chibon 300c54
   git format-patch -2
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
This will create two patch files that you can send by email to submit in a ticket
Pierre-Yves Chibon 300c54
on pagure, by email or after forking the project on pagure by submitting a
Pierre-Yves Chibon 300c54
pull-request (in which case the last step above ``git format-patch -2`` is not
Pierre-Yves Chibon 300c54
needed.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Unit-tests
Pierre-Yves Chibon 300c54
----------
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pagure has a number of unit-tests.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
We aim at having a full (100%) coverage of the whole code (including the
Pierre-Yves Chibon 300c54
Flask application) and of course a smart coverage as in we want to check
Pierre-Yves Chibon 300c54
that the functions work the way we want but also that they fail when we
Pierre-Yves Chibon 300c54
expect it and the way we expect it.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Tests checking that function are failing when/how we want are as important
Pierre-Yves Chibon 300c54
as tests checking they work the way they are intended to.
Pierre-Yves Chibon 300c54
rahul Bajaj c9695a
rahul Bajaj c9695a
So here are a few steps that one could perform to run unit-tests in a
rahul Bajaj c9695a
local pagure instance.
rahul Bajaj c9695a
rahul Bajaj c9695a
* Install the dependencies::
rahul Bajaj c9695a
rahul Bajaj c9695a
     pip install -r tests_requirements.txt
rahul Bajaj c9695a
rahul Bajaj c9695a
* Run it::
rahul Bajaj c9695a
rahul Bajaj c9695a
     ./runtests.sh
rahul Bajaj c9695a
rahul Bajaj c9695a
Pierre-Yves Chibon 300c54
``runtests.sh``, located at the top of the sources, helps to run the
Pierre-Yves Chibon 300c54
unit-tests of the project with coverage information using `python-nose
Pierre-Yves Chibon 300c54
<https: nose.readthedocs.org="">`_.</https:>
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
.. note:: You can specify additional arguments to the nose command used
Pierre-Yves Chibon 300c54
          in this script by just passing arguments to the script.
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
          For example you can specify the ``-x`` / ``--stop`` argument:
Pierre-Yves Chibon 300c54
          `Stop running tests after the first error or failure` by just doing
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
          ::
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
            ./runtests.sh --stop
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
Each unit-tests files (located under ``tests/``) can be called
Pierre-Yves Chibon 300c54
by alone, allowing easier debugging of the tests. For example:
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
::
Pierre-Yves Chibon 300c54
rahul Bajaj 889973
  python tests/test_pagure_lib.py
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 52c076
or using:
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
::
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
  ./runtests.sh tests/test_pagure_lib.py
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
If you with to run a single test, you can use the following structure:
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
::
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
  ./runtests.sh <file>:ClassName.method</file>
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
For example:
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
::
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 52c076
  ./runtests.sh tests/test_pagure_lib.py:PagureLibtests_search_user.test_search_user_username
Pierre-Yves Chibon 52c076
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
.. note:: In order to have coverage information you might have to install
Pierre-Yves Chibon 300c54
          ``python-coverage``
Pierre-Yves Chibon 300c54
Pierre-Yves Chibon 300c54
          ::
Pierre-Yves Chibon 300c54
rahul Bajaj af6966
            dnf install python-coverage
rahul Bajaj af6966
rahul Bajaj af6966
          or
rahul Bajaj af6966
rahul Bajaj af6966
          ::
rahul Bajaj af6966
Pierre-Yves Chibon 300c54
            yum install python-coverage