This is a great idea. It makes much more sense to run your tests in the cloud th...

runeks · on July 14, 2016

> I don't think it can be done 'as-you-type' due to resource constraints. But I do think it can be done pre-commit.

When you rent a Google Compute Engine machine, it doesn't matter how much you use the CPU, the monthly cost is the same (roughly $30/vCPU/month). So you incur no additional cost by restarting tests after each input character. And if we can spread multiple users' tests out over multiple machine instances, the cost would remain constant per user, but their tests would execute in parallel on multiple instances.

sytse · on July 15, 2016

If you restart the tests it indeed doesn't matters less. Currently GitLab doesn't allow for an easy cancel and rerun I think. Also keep in mind that that testing can keep over 30 CPU's busy for 5 minutes. I think it makes more sense to start that when a human indicates it. But we can also consider 'restart as you type' and 'restart when you save'. Thanks for your suggestion.

tedmiston · on July 14, 2016

Please do! —GitLab user

sytse · on July 14, 2016

Thanks, good to hear you like it. But this is quite an involved project. So we want to do Automatic parallelization of tests first https://gitlab.com/gitlab-org/gitlab-ce/issues/3819

And it will need a client integrations and a way to make this 'hidden' repository and/or commit.

tedmiston · on July 15, 2016

I agree this is very / more valuable, for my use case at least. The discussion around creating the build file is interesting.

In Django for instance, it would be straightforward to automate generating the list of apps to test:

    {
      "name": "my django project",
      "tasks": [
        "python manage.py test myapp1",
        "python manage.py test myapp2",
        "python manage.py test myapp3",
      ],
      "labels":"..."
    }

Those app names can be pulled from Django's settings.INSTALLED_APPS with a little config to avoid testing third party apps.

It would be cool if the architecture allows for plugins like this.

sytse · on July 15, 2016

That would be nice! For now we're offering static .gitlab-ci.yml templates for many languages in the interface itself so you can easily start. Making plugins is a larger undertaking.