Fix theta2 notebook

[mdebony]

This mostly fix issue #1353 .
Included change :

• Fix computation mistake on the off counts
• Fix computation mistake on the error bars for off counts
• Add an option to exclude events from on region (deactivated by default)

With the Crab example dataset from the school significance vary :

• 10 GeV - 100 GeV : Go down from 18 sigma to 17.8 sigma
• 100 GeV - 1TeV : Go down from 80 sigma to 74.5 sigma
• 1 TeV - 10 TeV : Go down from 69.7 sigma to 62.9 sigma

The alpha value is correct for 10 GeV - 100 GeV range (0.33) but still incorrect for 1 TeV-10 TeV range (0.17). This is very likely due to the contamination of the off counts by the Crab (visible in the shape of the off counts).

The option of exclusion region is deactivated by default as it could cause some issue (see plots below) and didn't completely fix the remaining issue on alpha value mentionned above.

I could look to implement a correction for the counts for the exclusion region, hoping to fix completely the issue, but I don't have an idea for a simplistic algorithm (in the idea of this notebook) right now.

Theta 2 plots without exclusion

Theta 2 plots with exclusion

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.86%. Comparing base (8c8f7f1) to head (b885581).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1354   +/-   ##
=======================================
  Coverage   72.86%   72.86%           
=======================================
  Files         137      137           
  Lines       14515    14515           
=======================================
  Hits        10576    10576           
  Misses       3939     3939           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[mdebony]

To add on the question of contamination. This is without exclusion but with a much larger theta2 plot and results normalize for theta2 > 1.0. alpha is now around 0.33 for all energy bin. For the two highest energy bin, the contamination of the off is quite easy to spot.

[moralejo]

This mostly fix issue #1353 .

The alpha value is correct for 10 GeV - 100 GeV range (0.33) but still incorrect for 1 TeV-10 TeV range (0.17). This is very likely due to the contamination of the off counts by the Crab (visible in the shape of the off counts).

I do not understand - why can't the alpha value be computed purely "from geometry"? How does the value 0.17 come up? Is it from normalization of counts in some range (a bad idea, particularly for a bright source)

The option of exclusion region is deactivated by default as it could cause some issue (see plots below) and didn't completely fix the remaining issue on alpha value mentionned above.
I could look to implement a correction for the counts for the exclusion region, hoping to fix completely the issue, but I don't have an idea for a simplistic algorithm (in the idea of this notebook) right now.

I think the simplest and most general would be Monte Carlo - just throw random events with uniform distribution in the F0V (large enough to contain on and off) and counting how many end up in on and off.

[mdebony]

This mostly fix issue #1353 .

The alpha value is correct for 10 GeV - 100 GeV range (0.33) but still incorrect for 1 TeV-10 TeV range (0.17). This is very likely due to the contamination of the off counts by the Crab (visible in the shape of the off counts).

I do not understand - why can't the alpha value be computed purely "from geometry"? How does the value 0.17 come up? Is it from normalization of counts in some range (a bad idea, particularly for a bright source)

In my opinion, nothing is preventing that. It's just a different logic. The normalization on some range is also making sense, but the combination of PSF and small offset make this exercise complex.
And the 0.17 is effectively computed from the background normalisation in range. The issue is due to background contamination.

[mdebony]

I've added an option (activated by default) to normalise the background only base on geometry.

[mdebony]

I think the simplest and most general would be Monte Carlo - just throw random events with uniform distribution in the F0V (large enough to contain on and off) and counting how many end up in on and off.

I could try, but will only work for not too large offset.

[moralejo]

I've added an option (activated by default) to normalise the background only base on geometry.
But is this assuming the Off is not overlapping with exclusion regions?

[moralejo]

I think the simplest and most general would be Monte Carlo - just throw random events with uniform distribution in the F0V (large enough to contain on and off) and counting how many end up in on and off.

I could try, but will only work for not too large offset.

I do not know how the implementation would look like, but I think it would work for any offset, at least to compute the purely geometrical factor (solid angle ratio).
The problem - which goes beyond what this PR intends to fix - is that, in case the off is overlapping with exclusion regions, one can no longer assume that the detector acceptance for the background in the on and off regions is the same (e.g. the "valid" part of the off region may be in average at further offset w.r.t. to the on). Perhaps this is what you meant.

[mdebony]

The problem - which goes beyond what this PR intends to fix - is that, in case the off is overlapping with exclusion regions, one can no longer assume that the detector acceptance for the background in the on and off regions is the same (e.g. the "valid" part of the off region may be in average at further offset w.r.t. to the on). Perhaps this is what you meant.

Yes, if the offset is small enough and theta2 axis small enough the uniform distribution will do the jobs, but at larger offset or theta2, you will definitely need a model to correct. And I'm not sure we should add this in the notebook as it will complexify a lot the computation and user will have to control the quality of the model.

[mdebony]

I've added an option (activated by default) to normalise the background only base on geometry.
But is this assuming the Off is not overlapping with exclusion regions?

Yes, but for point like source and standard offset, the region close to theta2 = 0 should not be affected too much (except maybe at the lowest energy).