23andMe personal genetic testing

[imoldernu]

This Forbes article is worth reading...

23andStupid: Is 23andMe Self-Destructing? - Forbes

Really? In six months, a company choosing to work in a business in which it knows the FDA believes it has jurisdiction decided not to respond to the agency for six months? At a time when 23andMe was going to be launching an advertising campaign to try to sign up a million people to its service? At a moment when Anne Wojcicki, the company’s chief executive, was going to be on the cover of FastCompany talking how 23andMe is revolutionizing health care? And 23andMe thought the FDA was just going to, I don’t know, not notice?

Either 23andMe is deliberately trying to force a battle with the FDA, which I think would potentially win points for the movement the company represents but kill the company itself, or it is simply guilty of the single dumbest regulatory strategy I have seen in 13 years of covering the Food and Drug Administration.

for starters... the rest of the article gives details...

 

[Katsmeow]

For another point of view, this article points out that the FDA action is basically a paternalistic course of action that is specifically designed to kill of the personal genetic testing industry. Apparently 23andme was the last man standing as all the others had already given up.

23andMe FDA letter: Premarket approval requirement could kill at-home genetic testing industry.

The interesting thing argued is that basically what the FDA is requiring 23andme to do to get its tests approved is not being required of physicians who order the same tests. That is, if you ask a physician to have you tested as to whether you have a certain gene the same type of test would be done. The difference is that (1) buying these tests one at a time through a physician is hideously expensive (a few years ago I had a genetic test done for my son and it cost several hundred dollars for just the one test) and (2) if a physician does the test the results will go into your medical record and will then be available to insurers.

I had been on the fence about doing 23andme (I am adopted and have health history information for my mother but not for the paternal side), but was leaning toward doing it. It is annoying to me in the extreme to have the FDA deciding that I'm basically too stupid to be able to handle the information about my genes.

 

[clifp]

Two interesting articles. My inclination is to side with 23andMe, like the Forbes author suggests. It would a shame if the FDA strangled this promising technology because it doesn't conform to some archaic rules.

On the other hand there has been a ton of hoaxes in the medical field. Wasn't it dried apricot that were suppose to cure cancer.

I certainly have no idea if the result 23amdMe would sent me are my actual DNA being sequenced or simply a random result like an astrology estimate.

So I think it is entirely reasonable for the FDA to demand that test are reproducible and can be otherwise replicated by scientist and medical researchers.

 

[robnplunder]

For another point of view, this article points out that the FDA action is basically a paternalistic course of action that is specifically designed to kill of the personal genetic testing industry. Apparently 23andme was the last man standing as all the others had already given up.

23andMe FDA letter: Premarket approval requirement could kill at-home genetic testing industry.

The interesting thing argued is that basically what the FDA is requiring 23andme to do to get its tests approved is not being required of physicians who order the same tests. That is, if you ask a physician to have you tested as to whether you have a certain gene the same type of test would be done. The difference is that (1) buying these tests one at a time through a physician is hideously expensive (a few years ago I had a genetic test done for my son and it cost several hundred dollars for just the one test) and (2) if a physician does the test the results will go into your medical record and will then be available to insurers.

I had been on the fence about doing 23andme (I am adopted and have health history information for my mother but not for the paternal side), but was leaning toward doing it. It is annoying to me in the extreme to have the FDA deciding that I'm basically too stupid to be able to handle the information about my genes.

I smell a strong lobby from medical industry on this issue. Oh, well. I just got an e-mail from 23andMe saying that the kit was sent to me. I believe I paid a few days before the FDA letter came out.

 

[Katsmeow]

So I think it is entirely reasonable for the FDA to demand that test are reproducible and can be otherwise replicated by scientist and medical researchers.

It is, but the problem is that the way the FDA wants them to prove it is outdated and not really designed for the modern DNA testing. To individually do the extensive tests the FDA wants for each DNA result would apparently take years to do, by which time the presumably now approved test would be outdated by even newer and better tests.

And for me the thing that makes me feel that the FDA isn't all that motivated by the above is the FDA isn't requiring such testing for all the DNA testing being ordered by physicians. Apparently, the FDA is only requiring such testing if the DNA testing is being sold to the consumer directly rather than through a doctor.

So if I go and pay $400 to get one gene tested for (as opposed to 23andme's $99 charge for testing a bunch of them), the FDA is AOK with that since a doctor ordered it even if no "testing" of the test has been submitted to anyone.

 

[MissMolly]

Email received from 23AndMe

I just received this email:

Dear 23andMe Customers,

I wanted to reach out to you about the FDA letter that was sent to 23andMe last Friday.

It is absolutely critical that our consumers get high quality genetic data that they can trust. We have worked extensively with our lab partner to make sure that the results we return are accurate. We stand behind the data that we return to customers - but we recognize that the FDA needs to be convinced of the quality of our data as well.

23andMe has been working with the FDA to navigate the correct regulatory path for direct-to-consumer genetic tests. This is new territory, not just for 23andMe, but for the FDA as well. The FDA is an important partner for 23andMe and we will be working hard to move forward with them.

I apologize for the limited response to the questions many of you have raised regarding the letter and its implications for the service. We don't have the answers to all of those questions yet, but as we learn more we will update you.

I am committed to providing each of you with a trusted consumer product rooted in high quality data that adheres to the best scientific standards. All of us at 23andMe believe that genetic information can lead to healthier lives.

Thank you for your loyalty to 23andMe. Please refer to our 23andMe blog for updates on this process.

Anne Wojcicki
Co-founder and CEO, 23andMe

 

[palomalou]

I did not receive this e-mail. Does it mean that those of us who paid and returned our kits will not receive any info, I wonder?

 

[martyp]

I ran the engineering group at DNA sequencing facility. Everyone submitting data to the Human Genome project database had to meet a minimum data quality standard. The newest DNA sequencing technologies have only been in production use for about 3 yrs. I have to believe that data quality standards within the research community has been agreed upon by now. However, data standards for medical use may be more stringent. As long as 23andMe has only been giving out ancestry information then they were OK. If they are going to give out genetic medical advice then it better be as good as what you get from your doctor. That is, if a doctor is contracting out for DNA sequencing is the data better or the same as what you get from 23andMe? Would a doctor submit samples to 23andMe?

Interpreting genetic results to make medical decisions is not straightforward. Lot's of medical medical diagnosis and treatments benefit from consultation with medical experts.

 

[photoguy]

I just received this email:

Dear 23andMe Customers,

I wanted to reach out to you about the FDA letter that was sent to 23andMe last Friday.

It is absolutely critical that our consumers get high quality genetic data that they can trust. We have worked extensively with our lab partner to make sure that the results we return are accurate.

Has 23andme ever published error rates? The last time I looked I didn't find any but they should have this data.

 

[martyp]

Has 23andme ever published error rates? The last time I looked I didn't find any but they should have this data.

I believe the Human Genome Project used a Phred score of Q40 which means 99.99% base pair accuracy. This was for sequencing a human genome for the 1st time.

I don't know what the data standards are now. For a company like 23andMe they have a standard reference Human Genome against which they can compare each sample. They also only have to process samples for one type of organism (Humans) so their is less variation in their production process than would be for a case like a research lab. Finally, the new sequencing technology, allows for massive amounts of data so they can achieve high redundancies. I would expect it to be higher than 99.99%.

Also, they may only have to be accurate in detecting known genes. This would be easier than base by base accuracy. You can have a lot of base pair errors and still identify the genes to essentially 100% accuracy.

In the end the test is just common sense. Find sections of the human genome whose sequence you know very accurately and sequence it a bunch of times until you get it right most of the time. Then implement and maintain a quality monitoring program.

 

[clifp]

It is, but the problem is that the way the FDA wants them to prove it is outdated and not really designed for the modern DNA testing. To individually do the extensive tests the FDA wants for each DNA result would apparently take years to do, by which time the presumably now approved test would be outdated by even newer and better tests.

And for me the thing that makes me feel that the FDA isn't all that motivated by the above is the FDA isn't requiring such testing for all the DNA testing being ordered by physicians. Apparently, the FDA is only requiring such testing if the DNA testing is being sold to the consumer directly rather than through a doctor.

So if I go and pay $400 to get one gene tested for (as opposed to 23andme's $99 charge for testing a bunch of them), the FDA is AOK with that since a doctor ordered it even if no "testing" of the test has been submitted to anyone.

Yes that makes sense..

After doing Angel investing for 7 years now and seeing scores of presentation on medical devices (stents and such), diagnostic tools (hi tech hospital beds), vaccines and several other categories. I've have a few observations.

1. The FDA generally appears to genuine in helping companies bring their products to market
2. The shear quantity of rules and regulations are mind boggling and have resulted in most small companies doing their preliminary testing overseas often Asia but sometimes Europe
3. Human clinical trial represent and almost insurmountably barrier for a new company to get through. Even one as well backed as 23andMe.
4. The more innovative the product the harder it is for the FDA to adapt. Which is why it is relatively (only a billion or so ) easy to bring on the market a new product or medical device that is only marginally superior to it replacements. This really handicaps small firms.

IMO, we have passed the point were we are letting our search for the perfection is getting in the way of the good medicine. But you can't only blame on the FDA, out legal system is a big factor.




All that being said it seem foolish in the extreme for 23andme to blow off the FDA for 6 months...

 

[martyp]

Does it really take 6 - 8 weeks for the results to come back? That's a long wait ....

We were using Illumina sequencers where I worked. A run of many samples took about 10 days. It varies depending on your protocol. Depending on the number of machines, the size of the input queue, and the machine downtime plus mailing, handling, and data analysis I think 6-8 weeks is a reasonable customer quote. I wouldn't be surprised if results often came back sooner.

 

[photoguy]

Also, they may only have to be accurate in detecting known genes. This would be easier than base by base accuracy. You can have a lot of base pair errors and still identify the genes to essentially 100% accuracy.

Thanks for your comments. I can certainly see how at the level of an entire gene, individual errors in base pairs don't matter much because a classifier would use the entire sequence as a input signal.

I guess what I don't understand is how the base-pair accuracy translates in errors at the level of gene mutation which I presume would involve a much smaller number of base-pairs (than a full gene). For example, in detecting the BRCA mutations that the FDA complained about in their letter.

 

[omni550]

UPDATE. The remaining balance of my results were posted on the 23andme site yesterday. Bottom-line, 5 weeks from mailing in the saliva sample to having complete test results posted.

Lots to read through and process.

Based on their ancestry results, it looks likely that I might have a 2nd cousin (we'd share a great grandparent) who is 3 years older than me in Germany. And their website easily allows for sending an email of introduction to this currently-anonymous person.

omni

 

[martyp]

I guess what I don't understand is how the base-pair accuracy translates in errors at the level of gene mutation which I presume would involve a much smaller number of base-pairs (than a full gene). For example, in detecting the BRCA mutations that the FDA complained about in their letter.

This is a general principal in measurement science - every measurement has error. You get past this by making the measurement multiple times until the probability that you are accurate exceeds some threshold limit. Are you comfortable with a result that has a 90% probability of being accurate? or does it have to be 99.99%. It will depend on what you intend to do with the information.

There is some inherent instrument accuracy associated with making a single base pair measurement. Typically though, DNA sequences are sequenced many times over and this is known as the "coverage". 50x coverage means that base has been sequenced 50 times.

If you are looking for a mutation in a short region of DNA sequence. Say a 1 base pair change in a region of 50 bps then you just have to sequence it a lot of times until the probability of being right is high enough to reach your comfort level of accuracy.

As a practical matter you are not actually sequencing the same physical piece of DNA over and over. They are looking at lots of identical DNA strands from the same individual and looking at the same region.

So, with regard to the FDA, I would surmise that they want to know what minimum standards and protocols are being used and how the quality assurance program being employed. I used to tell my operators that when they walk up to the machine that they should always ask themselves, "how do I know this machine is working correctly today"? You have to have some physical basis for knowing that the results the machine gives me are results I can believe for the intended use of the data.

 

[photoguy]

This is a general principal in measurement science - every measurement has error. You get past this by making the measurement multiple times until the probability that you are accurate exceeds some threshold limit.

Sure this makes sense if the errors are unbiased.

Are you comfortable with a result that has a 90% probability of being accurate? or does it have to be 99.99%. It will depend on what you intend to do with the information.

In this case, since 23andme is selling to the general public as a consumer service the final accuracy has to be very high otherwise multiple testing will result in too many false positives.

You have to have some physical basis for knowing that the results the machine gives me are results I can believe for the intended use of the data.

What I want to know is whats the "all-in" error rate accounting any type of errors in the process (whatever they may be).

 

[robnplunder]

UPDATE. The remaining balance of my results were posted on the 23andme site yesterday. Bottom-line, 5 weeks from mailing in the saliva sample to having complete test results posted.

Lots to read through and process.

omni

I've just sent ours (me & DW) in. I wonder if they are going to post partial results, none, or all, given the FDA letter.

 

[martyp]

Sure this makes sense if the errors are unbiased.



In this case, since 23andme is selling to the general public as a consumer service the final accuracy has to be very high otherwise multiple testing will result in too many false positives.



What I want to know is whats the "all-in" error rate accounting any type of errors in the process (whatever they may be).

There is a lot of experience interpreting sequence data. There are various biases but they can be delt with. Some areas of a genome resist being sequenced but I'm sure whatever they are doing at 23andMe is pretty straight forward.

Regarding the "all-in" error rate - that, I assume, is what the FDA wants to know in order to make sure that the use the public makes of the results is commensurate with the accuracy of the data. I wouldn't be surprised if 23andMe keeps the details secret from the public as proprietary data but they can share it with the FDA and the FDA would then give it's stamp of approval.

 

[palomalou]

The posts above are way beyond my limits of understanding without more time than I have now! However, I did get results and have SLIGHTLY elevated risk for something I feared, but now I know what to look out for. And it's not extreme.

 

[calmloki]

Was late ordering kits, but got them, just now got this email from 23andme:

"At this time, we have suspended the health-related part of our service to comply with the U.S. Food and Drug Administration’s directive on November 22, 2013 to discontinue new consumer access during our regulatory review process.
As a result, this means we will not be able to provide you with access to the 23andMe health-related results from your purchase.
We will be able to provide you with both ancestry-related genetic information as well as your raw genetic data, without 23andMe’s interpretation".

They are also offering a refund, which is nice - wish I'd been a bit quicker, was thinking this would be a nifty Christmas gift....

 

[robnplunder]

I've got the following e-mail from 23andMe. I am not asking for refund since my interest was mostly on raw/ancestry genetic info.


Dear 23andMe Customers,

Thank you for your recent 23andMe purchase.

At this time, we have suspended the health-related part of our service to comply with the U.S. Food and Drug Administration’s directive on November 22, 2013 to discontinue new consumer access during our regulatory review process.

As a result, this means we will not be able to provide you with access to the 23andMe health-related results from your purchase.

We will be able to provide you with both ancestry-related genetic information as well as your raw genetic data, without 23andMe’s interpretation.

We sincerely apologize for the inconvenience.

We understand this is not the product you purchased. If you would like a full refund for your order, we encourage you to click here and follow the on-screen instructions.

 

[braumeister]

Another update via email. Here is the relevant portion:

If you are a customer whose kit was purchased before November 22, 2013, your 23andMe experience will not change. You will be able to access both ancestry and health-related information as you always have.

23andMe has complied with the FDA's directive and stopped offering new consumers access to health-related genetic results while the company moves forward with the agency's regulatory review processes. Be sure to refer to our 23andMe blog for updates.

 

[robnplunder]

Just got DW's ancestry only results. For me, the information is worth the money. I am waiting for my results to be posted. Can't wait ....

 

[martyp]

Just got DW's ancestry only results. For me, the information is worth the money. I am waiting for my results to be posted. Can't wait ....

That's encouraging. I just sent in a DNA kit from Ancestry.com. I'm looking forward to the results.

 

[robnplunder]

Just got mine. DW's Neanderthal percentage = 2.7%. Mine, 2.8%. She's been making fun of it all day. Drat.