This tab is the repository of the various Divine Engine backtests that you did for this strategy (either a local backtest or with The Boss). Here you can see the graphical progression of each backtest, and choose the top strategies to use. In previous versions of Portfolio Boss, this is called the “Backtest Results” Tab. It's located at the bottom-mid of the Backtest Strategy Page.
This tab is divided into three sections:
This section lists the Backtest Items for this strategy. A Backtest Item is a single Divine Engine backtest, one created each time you start the Divine Engine backtest (either or ). So if you did five Divine Engine backtests, for example, there are five Backtest Items shown here:
Look at the info-bar on top “Showing all records. (5)”: the number in parentheses shows you the total Backtest Items you have. You can use the scrollbar (or mouse wheel) to see them all.
Each Backtest Item, in turn, contains its own top strategies that you can load and use. Simply click a Backtest Item, and its contents (top strategies) are shown in the section beneath it.
There are various columns for this section, explained below:
1. Favorite – This column allows you to mark a Backtest Item(s) as favorite.
Let's say you did a lot of Divine Engine backtests, and found some of them to be particularly interesting. You can mark them by clicking the star buttons here, so they don't get lost in the myriad items you have. The favorite Backtest Items are highlighted in blue:
You can also see just the favorite items (instead of all Backtest Items being shown): simply click the dropdown button (on this column's header), and a pop-up toggle appears:
If it's toggled OFF, all Backtest Items are shown, and the info-bar at the top says “Showing all records”. But if it's toggled ON, only the favorite items are shown, and the info-bar says “Favorites filter enabled”. The info-bar is there so you don't get confused on what you're seeing; there could be a case where the list is empty because you haven't starred an item, yet you set this toggle ON. If you really don't have a single Backtest Item, the info-bar will say “No results yet.”:
2. Type – This shows the type of Divine Engine backtests that you did.
It can be either (The Boss) or (Local Test).
3. State – This shows the status of each Divine Engine backtest.
There are seven possible states:
Note, for a cloud backtest, even if there's an error that impedes your backtest process, The Boss will continually update itself for a fix of such problem. If the problem is resolved and The Boss has finished synchronizing with it, your backtest will be continued to the finish.
You might want to check the Portfolio Boss Status Page if there are indeed problems with The Boss service:
4. Date Started – This shows the date and time the backtest was started (submitted, for cloud backtests).
It's the local date & time on your computer. All Backtest Items are sorted by this date & time, with the latest at the top.
5. Memo – This column shows the memo (description) for each Backtest Item. It also allows you to add (or edit) such memo.
Such memo could be written on the backtest confirmation dialog (when you started the backtest), or here post-backtest with the button as shown above. Click that button and a text field appears:
Once done, click “Save Changes”. Note that memos are optional, you can leave them empty.
6. Cancel Button – This button is only visible if a Divine Engine backtest is running. It's used to cancel a cloud backtest.
There's no confirmation dialog when you press that button. The cloud backtest's status immediately changes to . You may want to cancel a backtest if you realize its settings are wrong (e.g. wrong Fitness Functions criteria); make sure you cancel it early, before your compute credits are expended much.
If you run a local backtest (Random search) use the cancel button on its progress dialog:
Once a backtest is successfully canceled, its state will be .
7. Load Start Parameters – These buttons let you load the Baseline Strategy.
Each Divine Engine backtest has a Baseline Strategy, i.e. the initial strategy (along with its Divine Engine settings) that will spawn populations and generations (or derivative iterations). It's the original strategy created by the user, before the Divine Engine backtest is initiated. Even if you didn't add any rules (i.e. create from scratch), the Baseline was that blank strategy without any rules (but System Setting's rules must still be present).
So, if you initiate a Divine Engine backtest based on a top-strategy you loaded (without customizing it), the Baseline would be exactly the same as that top strategy.
8. Delete Buttons – These buttons delete the Backtest Items.
Deleting a Backtest Item also erases its top results, essentially erasing that Backtest Item from this strategy's history. This is good if you want to remove trash or faulty backtests. There's a confirmation dialog after you press that button:
To remove multiple Backtest Items at once, select them (by ticking their checkboxes) and press the button at top right corner:
This section lists the top strategies (of a Backtest Item you selected). You can also load a top strategy so its rules & contents are applied as the active strategy. This list (and the chart below it) are compiled even as the backtest is still in progress.
Let's go through the columns there:
1. Rank – This column shows the rank of each top strategy.
The ranking is based on their in-sample Fitness Score, with the highest score given the number 1 rank.
2. Load Parameters – These buttons load the parameters of a top strategy.
In effect, you load that strategy's rules, filters, parameter values, portfolio, Divine Engine settings (on its Backtest Parameterization Panel), and even the original name it was backtested with (the Baseline Strategy's original name). Thus it becomes the active strategy.
Note while a cloud backtest is still running, you can load its top strategy and do a standard (manual) backtest to populate its reports.
3. Description – This shows the generation from which the top strategy was pulled.
For example you see “Gen. 035”, which means the strategy was pulled from the 35th Generation. If you use Random search instead of Evolution, this column shows “Iteration” instead of Generation:
Note, if you have a valid Baseline Strategy, it may show here as “Baseline”, whether it's a winning strategy or not.
4. Fitness – These two columns show the Fitness Score of the top strategies, both in-sample (IS) and out-of-sample (OOS) score.
The Fitness Score is based on the Fitness Functions (metrics) you set for each Divine Engine backtest. The score of all metrics are aggregated (and normalized into a factor of 1) to yield this final Fitness Score. With or without the Rotational Fitness, the score displayed here is the aggregate of all Fitness Functions used.
Fitness (IS) comes from the metrics' score during the in-sample period, while Fitness (OOS) comes from the out-of-sample period. Make sure you select a top strategy whose IS and OOS Fitness Score are generally similar.
5. Metrics Score – These columns show the actual metrics' score of the top strategies.
They're useful to see the individual metric's performance, unlike the aggregate Fitness Score. Even if a strategy has a good Fitness Score, some metrics may be lacking, and you may decide to discard it. By default, any metrics listed as Fitness Functions (on the Backtest Parameterization Panel) will have their columns shown here.
Sometimes you'll see columns of other metrics not part of the Fitness Functions; these are simply residue from the latest Divine Engine backtest. You can use the “Manage Columns” dialog to define which metrics have their columns shown; click the button as shown here:
Here you can search & select any metrics, either their in-sample (IS) score, out-of-sample (OOS), or all sample (AS):
To hide the columns again, simply deselect them from this dialog and press OK. Note that any metrics you added on the Fitness Function section (of the Backtest Parameterization Panel) are grayed out (with italics).
Their columns (IS and OOS) are always shown and can't be removed with this dialog, unless you delete the Fitness Function(s) first.
6. Test Date – This column shows the date and time each top strategy was born.
The time and date are based on your local computer's, not the cloud.
7. The Delete Buttons – These allow you to delete top strategies.
Clicking them opens a dialog to confirm deletion:
Once removed, it's gone entirely from this strategy's history. But the Fitness Series Chart stays unchanged, as it's supposed to tell you what happened during the backtest.
This is useful to delete garbage results, so your collection of top strategies stays neat and easy to find.
This chart is the graphical representation of the evolutionary progression; if you use Random search, this chart won't be populated. Each Backtest Item (cloud) has its own distinct chart, and it's continually updated as the Divine Engine backtest is progressing.
The horizontal axis represents the generations, going from the earliest generation at the left, toward the latest generation at the right.
The maximum generation is dictated by the Max. Generations property at the Backtest Parameterization Panel. This may not be reached if an Exit Condition is triggered. If such is the case, the generations shown in this chart may be less than what you specified.
The vertical axis represents the Fitness Score, normalized into a factor of 1.
Note, there's an inset at the top-left corner. It allows you to see which graph belongs to what color, as well as enabling or disabling certain graphs (by ticking them on/off). You can expand or collapse the inset by clicking its arrow button. The “Generation Strategy Fitness” IS and OOS lines (purple and dark blue) are disabled by default. Let's discuss the six graphs (lines) there:
Throughout the course of the evolution, the dark-blue line may join the orange line, indicating that a new, offspring strategy, has beaten a parent strategy carried over from previous generations (orange line). It means that the best population strategy (orange line) comes from the best of the offspring (dark-blue).
While the evolution is progressing, you may want to take a look at each of these graphs. If they show abnormalities, you should cancel the backtest altogether, even before the Exit Condition is hit. For example, if the brown and orange lines show great divergence (obvious disparate values throughout the generations), you know that the top strategies produced have inconsistent performance. Ditto if the green and cyan lines are separated too much (make sure you look at the vertical scale too, so as not to get confused whether the distance is actually big). Also, if the cyan line does not advance (even declining) throughout the generations, you know the evolution is probably useless.