The default configuration for NCrunch tends to favour compatibility over performance. Once you are familiar with the basic workings of the engine, it is worthwhile to make an effort to tune NCrunch more specifically for your hardware and the projects you are working on.
All configuration options for performance tuning can be found in the
component configuration and
global configuration.
Optimising CPU UtilisationThe options in the global configuration focus on settings that are specific to the machine you are using. Performance wise, the most important options to adjust are the CPUCoresAssignedToNCrunch and CPUCoresAssignedToVisualStudio. These options control how NCrunch distributes its processing load between your CPUs. Assigning more cores to NCrunch will allow it to process its
test pipeline faster, but beware of taking too much CPU away from Visual Studio as this may cause the IDE to act sluggish and jerky.
When assigning a CPU core to be used for a Visual Studio or the background processing, NCrunch will hard lock the CPU at kernel level to only process the work it has been assigned to do (within the scope of the VS and NCrunch processes). The major benefit of this is that a certain percentage of your overall CPU time can be reserved entirely for handling actions within the IDE without ever being interrupted by NCrunch's background churning.
The optimal configuration of your CPU cores will depend entirely on your situation. If you are operating within a large solution and you are also using Jetbrains' Resharper with solution-wide analysis enabled, it is recommended that you treat Visual Studio as a first class citizen and provide it with at least two processor cores (if possible). If you are working on smaller projects and are not using real-time code analysis tools, one CPU core should be enough to give optimal performance.
It is strongly recommended that you try to avoid assigning CPU cores to both Visual Studio
and NCrunch. This can cause excessive purging of CPU cache as processor cores flick between tasks, and it will likely impact the performance of your IDE.
Optimising Processing TimesReducing processing times is all about
parallel execution of tasks in the
processing queue.
The MaxNumberOfProcessingThreads
configuration option controls the number of background tasks that can be executed by NCrunch at one time (as visible in the
processing queue).
Under normal circumstances, you should set this value to be equal to the number of processing cores you have assigned to NCrunch. If you are running automated tests that do a large amount of blocking (I.e. on sockets, timers, database calls, etc), you may find it useful to set this value slightly higher - as there should be more CPU slack to use.
Note that setting this value too high can cause the CPU to become overloaded, causing excessive context switching and slowing down the cycle times of NCrunch's core engine. It's often worth running through a few engine cycles with a CPU performance monitor running in order to strike the right balance.
If you are experiencing issues with CPU-intensive tests running slowly under NCrunch, disabling performance profiling for projects that contain this code may solve this. You can disable performance profiling in NCrunch's
component configuration.
Optimising Memory UsageThe ProcessPoolSize
configuration option controls the size of the pool of worker processes used to perform background tasks.
NCrunch will generally kick off new processes as they are required, and will pool them when they are not being used. The pooling helps to ensure that NCrunch will not waste time spawning new .exes when existing processes already exist that can do the job.
NCrunch uses two process pools - one for building components, and one for running tests. Because NCrunch will always spawn enough processes to cater for all tasks being executed by its engine, the physical number of task runner processes will not be equal to the value set in the ProcessPoolSize although it will be affected by it.
If you are running some very heavyweight tests and you find your memory utilisation is too high, try turning down the MaxNumberOfProcessesInPool configuration option. If NCrunch is still using too much memory, you should reduce the MaxNumberOfProcessingThreads setting to a lower number to prevent the engine from kicking off too much work at the same time.
Optimising Build TimesNCrunch generally tends to behave better with solutions that are made up with a large number of small projects. This is because the
optimisations in the execution engine allow NCrunch to only build the components it actually needs to, rather than the entire set (or a large subset).
Creating solutions with large numbers of components has some definite drawbacks - foremost of which being that it can make a codebase harder to understand. This is something that should be balanced carefully, as having a large component with 200,000 lines of code could easily add a huge amount of time to any of NCrunch's engine cycles.
Unless it is absolutely required, ensure you set the CopyReferencedAssembliesToWorkspace
configuration option to
false. Leaving this setting turned on will force NCrunch to build a referencing assembly every time one of its dependencies is changed - greatly increasing the amount of work the engine needs to do. The copying of assemblies around the file system will also marginally increase build times for the component.
Optimising TestsWriting high quality automated tests has always been important, and working with NCrunch makes this even more so.
High quality tests should be:
- Atomic
- Repeatable
- Consistent
- Without side-effects
- Able to be paralleled
- FAST
The first 4 above points are a basic requirement of any automated test used regularly within a development process. Any tests that do not meet these four requirements you should deliberately
ignore or exclude from continuous execution (using a
custom engine mode) with NCrunch until they can be fixed up, or flag them with the
Isolated attribute so that they can be run in their own task runner process.
Tests that cannot be
paralleled are a performance constraint for NCrunch and must be marked using the
ExclusivelyUses and
InclusivelyUses attributes. Failing to do this will result in the tests interfering with each other and going haywire when NCrunch is set to run them in parallel.
The speed of tests under execution is always very important, as faster tests can be run more frequently and with a lower risk of blocking up the
test pipeline.
Generally speaking, high quality tests = a better NCrunch experience.
