Cancerguard: local and population level

[engineernerd]

6miths, are you sure you aren't confusing these blood tests with something else? CancerGuard reports specificity of 97.4% and Galleri 99.6%. Both have much lower sensitivity, that's their main shortcoming.

 

[Gary Smith]

25-50% of the positives are false positives. Could you imagine the havoc this would unleash if done on the entire population. The unhappiness and anxiety, the financial cost? My advice to healthcare students is never to volunteer for the 'free' MRI or whatever.

That pretty much sums it up. Thanks. One of many reasons why I am not into tests. I recall I had minor surgery 14 years ago. Had to have a chest X Ray. Surgery got put off for awhile and had to have another chest X ray. I was told something suspicious was going on and had better have some invasive scope put down my throat. I said thanks but no thanks. The doctor said are you just going to wait till you start throwing up blood and the like till you take action. I told him I felt fine and my running/hiking would alert me to any issues. Now 14 years later at 79 still running and hiking with no issues.

 

[target2019]

That pretty much sums it up. Thanks. One of many reasons why I am not into tests. I recall I had minor surgery 14 years ago. Had to have a chest X Ray. Surgery got put off for awhile and had to have another chest X ray. I was told something suspicious was going on and had better have some invasive scope put down my throat. I said thanks but no thanks. The doctor said are you just going to wait till you start throwing up blood and the like till you take action. I told him I felt fine and my running/hiking would alert me to any issues. Now 14 years later at 79 still running and hiking with no issues.

Glad it worked for you.

But many others did not, or do not, dodge the cancer bullet. One size does not fit all?

 

[Gary Smith]

Glad it worked for you.

But many others did not, or do not, dodge the cancer bullet. One size does not fit all?

I could not agree more with you. My way is definitely not the way. Have seen a dentist but once since 1969.

 

[sengsational]

6miths, are you sure you aren't confusing these blood tests with something else? CancerGuard reports specificity of 97.4% and Galleri 99.6%. Both have much lower sensitivity, that's their main shortcoming.

Specificity and sensitivity only tell part of the story. When you use a population with a rare condition, you still get a lot of false positives. I had the google machine calculate the positive predictive value if the expected rate in the population is 1%. That turns out to be about 18%, meaning out of 100 people tested, 82 people will get scary news, but not have cancer.

If you want to dig into it, ask your favorite AI to generate a 2 by 2 confusion matrix with 97% specificity, 64% sensitivity (Cancerguard stats). You can adjust the rate of occurrence in the population. If you had a family history of cancer, that 1% is probably too low. If you're a higher risk population, say 5%, the PPV would be 53%. Still, about a 50-50 chance that the bad news not cancer.

If you don't mind spending money on imaging to rule-out the cancer, and you don't mind the stress associated with a period where you're worrying about having cancer, it might be something to do. Not for everyone, certainly. Another reason to do it is to reduce regret; if you do end-up with cancer later that's caught too late to do anything with and you didn't do this "when you had the chance", that might be a tough thing to live with.

 

[engineernerd]

Specificity and sensitivity only tell part of the story. When you use a population with a rare condition, you still get a lot of false positives. I had the google machine calculate the positive predictive value if the expected rate in the population is 1%. That turns out to be about 18%, meaning out of 100 people tested, 82 people will get scary news, but not have cancer.

If you want to dig into it, ask your favorite AI to generate a 2 by 2 confusion matrix with 97% specificity, 64% sensitivity (Cancerguard stats). You can adjust the rate of occurrence in the population. If you had a family history of cancer, that 1% is probably too low. If you're a higher risk population, say 5%, the PPV would be 53%. Still, about a 50-50 chance that the bad news not cancer.

If you don't mind spending money on imaging to rule-out the cancer, and you don't mind the stress associated with a period where you're worrying about having cancer, it might be something to do. Not for everyone, certainly. Another reason to do it is to reduce regret; if you do end-up with cancer later that's caught too late to do anything with and you didn't do this "when you had the chance", that might be a tough thing to live with.

As I understand it, specificity of 97.4% (CancerGuard) means out of every 1000 positive tests 26 turn out to be false positives. Specificity of 99.6% (Galleri) means out of every 1000 positive tests 9996 are true positives and 4 are false positives. I don't think whether a condition is rare or not alters that math. Just means you have to run a lot of tests to get those 1000 positive tests. Or am I missing something?

 

[sengsational]

As I understand it, specificity of 97.4% (CancerGuard) means out of every 1000 positive tests 26 turn out to be false positives. Specificity of 99.6% (Galleri) means out of every 1000 positive tests 9996 are true positives and 4 are false positives. I don't think whether a condition is rare or not alters that math. Just means you have to run a lot of tests to get those 1000 positive tests. Or am I missing something?

Yeah it's confusing. That's why they call the 2 by 2 matrix the confusion matrix. The thing that matters is the predictive value, and the inputs to that include specificity, sensitivity and an estimate of the true rate in the population. I don't think they covered this in my engineering statistics class (or if they did, it didn't stick). But I have looked at enough examples to get it. If you put in 97% specificity, 64% sensitivity and 1% incidence into AI, look at the numbers in the matrix. And run it with a 5% incidence. And see how the positive predictive value is calculated. Maybe Galleri will look better, but I don't know about the sensitivity, which makes a big difference. On my phone right now, so not easy to research.

 

[hesperus]

I'm not expert on it, but I have popped into the CancerGuard web site and had a discussion with my wife about it.

My wife saw ads for this while watching her NCAA men's basketball and asked me about it. I might have heard Peter Attia talk about cancer screening but that was imaging cancer studies, where you go get scans and they look for signs of cancer. This is a blood test.

It's not covered by insurance because they only pay after you're having symptoms. So on a personal level, do you drop the $700 and find out if you have one of the 15 or 20 cancer types this test detects, or do you wait for symptoms?

If the test is positive, you're still on the hook to pay for the scans, not covered by insurance. If you get bad news (yes, you have cancer), only then does insurance kick in, after you have your diagnosis.

On a personal level, it would seem to be a good idea, if you could afford it, to get the bad news earlier, when treatment might actually be completely effective. Depending on the kind of cancer, I suppose, waiting for symptoms might make it impossible to eradicate.

On a population level, if everybody got the test, there would certainly be a huge number of people getting treated earlier. So a big increase in treatment services. Having talked about the big increases in Medigap premiums, I even wondered if the insurance companies are already seeing this increase.

I didn't see a topic on this, and it seems like something to be pondered.

yeah the idea sounds good on paper but it’s not that simple… early detection helps, but these blood tests can have false positives and false negatives, so you might end up stressed or paying for follow ups that find nothing, or worse get false reassurance, also they don’t replace real screening like colonoscopy, mammogram etc, those are still the gold standard for specific cancers, if you can afford it it might be an extra layer, but I wouldn’t treat it like a must have or a substitute for proper checkups, more like optional info with limits

 

[IncomeOriented]

Our Primary Care Provider, PCP., suggested the blood tests for me and wife. PCP was very clear that it is new and still somewhat experimental. We decided to do it. DW tests came up with nothing. My tests indicated a colonoscopy. Never had one before, but OH WELL, here we go.

Just had it done, and Doc removed 2 small polyps. Doc was generally unconcerned about it. Now we are waiting for the definitive lab results.

If we found nothing, that's great. If we find something, it's early detection.

 

[sengsational]

If we found nothing, that's great. If we find something, it's early detection

With some cancers, early detection can be the difference between a long and a short life. The big payoff would be you got a positive, went for imaging that you would not have done otherwise, that was positive and you had an early intervention before metastasis.

My wife's PCP said she didn't know enough about it to comment...not very helpful!

 

[engineernerd]

Specificity and sensitivity only tell part of the story. When you use a population with a rare condition, you still get a lot of false positives. I had the google machine calculate the positive predictive value if the expected rate in the population is 1%. That turns out to be about 18%, meaning out of 100 people tested, 82 people will get scary news, but not have cancer.[snip]

I dunno, that negates a simple definition. If it is true, then the word "specificity" doesn't mean what it says it does. Twilight Zone territory to me.

 

[walktalk]

Honestly, if $700 isn’t a big hit for you, I’d probably do it just for peace of mind - but I’d go in knowing false positives can send you down a stressful (and expensive) rabbit hole.

 

[engineernerd]

Honestly, if $700 isn’t a big hit for you, I’d probably do it just for peace of mind - but I’d go in knowing false positives can send you down a stressful (and expensive) rabbit hole.

These tests seem to mostly catch cancers in Stage 3, they aren't sensitive enough to catch much cancer in Stages 1 or 2.

So to me what makes sense is to either skip these tests until they improve enough to catch cancer earlier, or become a lot cheaper, or both - or plan to test often enough so cancer can't progress all the way through Stage 3 in the time between tests.

 

[6miths]

Specificity and sensitivity only tell part of the story. When you use a population with a rare condition, you still get a lot of false positives. I had the google machine calculate the positive predictive value if the expected rate in the population is 1%. That turns out to be about 18%, meaning out of 100 people tested, 82 people will get scary news, but not have cancer.

If you want to dig into it, ask your favorite AI to generate a 2 by 2 confusion matrix with 97% specificity, 64% sensitivity (Cancerguard stats). You can adjust the rate of occurrence in the population. If you had a family history of cancer, that 1% is probably too low. If you're a higher risk population, say 5%, the PPV would be 53%. Still, about a 50-50 chance that the bad news not cancer.

If you don't mind spending money on imaging to rule-out the cancer, and you don't mind the stress associated with a period where you're worrying about having cancer, it might be something to do. Not for everyone, certainly. Another reason to do it is to reduce regret; if you do end-up with cancer later that's caught too late to do anything with and you didn't do this "when you had the chance", that might be a tough thing to live wit

6miths, are you sure you aren't confusing these blood tests with something else? CancerGuard reports specificity of 97.4% and Galleri 99.6%. Both have much lower sensitivity, that's their main shortcoming.

Thank you for taking this up for me sengsational. I am not confusing them with something else. As sengsational says, sensitivity and specificity are characteristics of a particular test. The actual predictive values are dependent on the population that the test is being used on as well. This area of epidemiology and statistics are not particularily intuitive. If one tests a population that has a relatively low incidence of a conditon (such as cancer in the general population) then even a very specific test will still have a fairly high false positive rate. The lower the prevalence of the the condition, the higher the false positive rate will be. The sensitivity of a test is equivalent to 'Positivity in Disease' - a 100% sensitive test will be positive in all people with the condition. The specificity of a test is equivalent to 'Negativity in Health'- a 100% specific test is negative in all people who do not have the condition. So in this sense the meaning of the word 'specificity' does not mean that the test is 'specific' for the condition, rather it means that it is 'specific' for the lack of condition in the those with a negative test. It is true that a 100% specific test would have no false positives, but that is not the case with these tests. And although 99% or 99.9% specificiity sound wonderful, and are in fact excellent, the false positive rates can be quite problematic when screening low prevalence conditions. As sengsational suggests, using a confusion matrix to examine the actual numbers can be very educational, and eye-opening.

 

[sengsational]

They way they tested cancerguard was they took a group of people known to have cancer, but not yet treated, and a group of known-healthy controls. This is legit to validate the sensitivity and specificity values, but it's really meaningless in real-world testing. Because their group had over 20% people with cancer, if they got bad news from the test, it was an 85% chance it was really bad news (positive predictive value).

using a confusion matrix to examine the actual numbers can be very educational, and eye-opening.

Because many of us here are over 65, I calculated the confusion matrix for that cohort. The bottom line is for every 10 people over 65 from the general population that get the cancerguard test, 7 will go through more testing to find no cancer (30% positive predictive value).

I put 2% of the seemingly healthy general population as the actual rate of cancer (200 out of 10000). The calculations are easy, so make your own matrix in a spreadsheet and fiddle with that if you want to.

The other numbers are based on cancerguard's published specificity (9545/9800) and sensitivity (111/200).

It's the yellow area that causes the extra testing and hand-wringing. You're 70% likely to be thrown into that situation.

If you don't mind paying, then you need to start thinking about a whole new aspect of the decision which is how likely is it that learning about the cancer earlier will make what's left of your life better. In other words, maybe they can't do much about it, in which case, you'd have been better off with your head in the sand. But maybe they CAN do something about it, and save you a lot of suffering and you access a longer lifespan / healthspan.

I understand why people wouldn't want to do this test because the "big payoff" is probably going to be surgery or chemo or some other unpleasantness. And you're 70% likely to just waste money and time getting scanned or biopsied.

 

[engineernerd]

OK, I looked it up and sengsational and 6miths are right that your odds of getting a false positive result are expressed by PPV (positive predictive value), not test specificity.

THE CLIFF-NOTES:

PPV looks at the share of all "positive" test results that are true positives, which is the relevant statistic.
Test specificity only looks at the share of tested cancer-free people who are really cancer-free.

THE DETAILS:

I had to look up the definitions and found this factsheet explaining them mathematically:

https://centerforhealthsecurity.org/sites/default/files/2022-11/201207-sensitivity-specificty-factsheet.pdf

test specificity (97.4% for CancerGuard) = TN / (TN+FP) ....where T=True, N=Negative,etc.
positive predictive value (PPV) = TP / (TP+FP)

Because TN, FP, and TP are in units of total number of tests, you can't calculate PPV from the known test specificity without also knowing the relative sizes of the populations that do and do not have cancer.

For a given test specificity, if only a few people in the demographic of interest actually have cancer, then both TN and FP will be larger, and TP (and FN) will be smaller, than if a lot of people in that demographic have cancer.

Basically, a small percent of a large number (FP) is still a large number when combined in a formula with a large percent of a small number (TP).

 

[engineernerd]

The prevalence of cancer, based on 2022 data, in the United States population seems to be 5.4%.

Per the The National Cancer Institute (link: Cancer of Any Site - Cancer Stat Facts), the number of Americans living with cancer in 2022 was 18,000,110. Wonder if they are really that sure about the 110, but whatever.

Different websites gave different numbers for the total population in 2022, but the estimates were around 333 to 334 million. So 18/333 = .054.

Probably the prevalence of cancer in older age groups is higher. On the other hand, the prevalence of the kinds of cancer that aren't already adequately screened for via colonoscopies and such, AND are really important to catch as early as possible, would be less.

 

[6miths]

Great discussion. Thanks for the acknowledgement engineernerd. Epidemiology and the stats involved are very difficult to get a handle on and even experienced clinicians can be unclear on things. I encourage people to look the Wilson/Jungner criteria for screening for diseases before getting too carried away. Occasionally, there is a game changing test, a recent example is in the fetal-DNA from maternal blood in the realm of prenatal screening and diagnosis, but most advances are incremental. Your point that any new screening intervention must be viewed in the context of screening that already exists is very valid as well.

 

[engineernerd]

Great discussion. Thanks for the acknowledgement engineernerd. [snip]

Thanks for the chance to learn something new and useful!

 

[sengsational]

The prevalence of cancer, based on 2022 data, in the United States population seems to be 5.4%.

I think the cancerguard test is applicable/advised for people who are not diagnosed with cancer. I'm not sure the percent of people with a diagnosis can be used to as-is to determine the underlying rate in the remaining undiagnosed population. It probably is a good upper bound, though (that's my unreasearched guess).

 

[engineernerd]

I think the cancerguard test is applicable/advised for people who are not diagnosed with cancer. I'm not sure the percent of people with a diagnosis can be used to as-is to determine the underlying rate in the remaining undiagnosed population. It probably is a good upper bound, though (that's my unreasearched guess).

I think the definition of prevalence for the PPV calculation is for the overall population including those known to have the disease, isn't it? I'd say the actual prevalence would be higher than what the diagnosed share is, since it would include undiagnosed cases as well.

In any case, with CancerGuard's reported test specificity of 97.4% and sensitivity of 64%, if cancer prevalence is 6%, then for every 100,000 people tested the math is:
6,000 sick people and 94,000 healthy people
0.974=TN/(TN+FP)=TN/94,000 so TN=91,556 and FP=2,444
0.64=TP/(TP+FN)=TP/6,000 so TP=3,840
PPV=TP/(TP+FP)=3,840/(3,840+2,444)=0.611

So 39% of test results would be false positives. If actual prevalence is lower, the false positives would be higher.

Combine that with the likely failure to detect much cancer at Stages I and II, and the relatively high cost of testing regularly (as would be necessary to get the benefit), and the case for these tests seems weak to me.

 

[sengsational]

What we really care about is "Am I walking around with a cancer that, if I knew the details about it, would take medical action right now that has a high chance of improving the remainder of my life?" For the undiagnosed 65+ crowd, that's probably between 3% and 6%, but the AI is being evasive (is telling me a bunch of stuff that I didn't ask about, as usual).

But yeah, it's a pretty weak case. Somewhere between 4 and, say, 6 out of 10 people will go through imaging and/or biopsy for nothing. Beyond just the cost, the testing is not completely risk-free.

Just thinking more about this, I wonder if a person's current overall health status should enter the equation.

If you're very creaky and have some known conditions beyond cancer that are probably going to kill you "pretty soon", there's less of an argument to be made to find an asymptomatic cancer. But if you're a 65+ that looks and acts 20 years your junior, it seems like a more powerful case can be made to take greater efforts to find and eradicate an undiagnosed cancer. That's probably at least some of the reason why there's a lot of disagreement about whether to pursue this kind of test or not.