UNITED STATES DISTRICT COURT NORTHERN DISTRICT OF CALIFORNIA SAN FRANCISCO DIVISION
April 17, 2013
MATTHEW EDWARDS, GEORGIA BROWNE, AND
TORAH MONTESSORI SCHOOL, INDIVIDUALLY AND ON BEHALF OF ALL OTHERS SIMILARLY SITUATED,
NATIONAL MILK FEDERATION AKA COOPERATIVES WORKING TOGETHER;
DAIRY FARMERS OF AMERICA, INC.;
LAND O'LAKES, INC.;
DAIRYLEA COOPERATIVE INC.; AND AGRI-MARK, INC.,
The opinion of the court was delivered by: Honorable Jeffrey S. White United States District Judge
CLASS ACTION JOINT STIPULATION AND PRODUCERS [PROPOSED] ORDER RE THE USE OF PREDICTIVE CODING TECHNOLOGY ACTION FILED: September 26, 2011
a Party) hereby stipulate, agree, and propose as follows: 3
documents collected from Land O'Lakes and have agreed that Land O'
Lakes will utilize Recommind's Axcelerate software for its review workflow and production
in this case. Axcelerate 15 incorporates predictive coding (also known
as technology-assisted review) functionality, which Land
O'Lakes will leverage for a more cost efficient and higher quality
review. The Parties agree that the 17 following protocol will be
followed to identify potentially responsive documents for review:
possession, custody, or control of each custodian agreed upon by
the Parties and in additional 20 locations under the possession,
custody, or control of Land O'Lakes (such as shared drives or 21
departmental files) immediately upon entry of this order and isolate
the document review corpus. 22
Land O'Lakes will generate an initial control set of documents. The control set will be a random, 24 statistically valid sampling of documents to estimate the number of responsive documents in the 25 corpus. The control set sample shall be determined using a 95% confidence level and 2% estimation 26 interval. The control set sample will then be reviewed for responsiveness and privilege, and Land 27 O'Lakes will provide Plaintiffs with a tracking report of the results of the control set generation and 28 review. At any point during the predictive coding process, Land O'Lakes' e-discovery liaison will
Plaintiffs and Defendant Land O'Lakes, Inc. (collectively "the Parties" and each, individually,
A. "Document review corpus" refers to the body of documents remaining after the exclusion of known system files, incompatible file types (e.g., database files), duplicates, and documents that fall outside the agreed date range.
B. "Confidence level" means the statistical reliability of a result and in this instance refers to the likelihood that a measurement reached through sampling is accurate.
C. "Estimation interval" refers to the statistical error rate of a measured confidence level.
The Parties have discussed the methodologies or protocols for the search and review of 1. Step One -- Document Collection: Land O'Lakes will collect documents in the 2. Step Two -- Control Set: After loading the document review corpus into Axcelerate, meet and confer at Plaintiffs' request to discuss the results and method of the process. Land O'Lakes 2 agrees to confer in good faith regarding the incorporation of any input from Plaintiffs on the 3 application process. 4
5 during review of the control set will be included in an initial seed set, which will be used to train 6
Axcelerate to identify other potentially responsive documents in the document review corpus. The 7 seed set also will include responsive documents identified by Land O'Lakes through the use of search 8 and analytical tools, which will include the following: 9
b) The 500 most highly ranked1 responsive documents hit upon by an application of the
Boolean search terms selected by Land O'Lakes (listed in Appendix A), excluding documents already contained in (a), above; and
c) The 500 most highly ranked responsive documents hit upon by application of Boolean search terms selected by Plaintiffs (listed in Appendix A), excluding documents already contained in (a) and (b), above.
Land O'Lakes will provide Plaintiffs with a search hit count of all searches conducted and will 17 produce all non-privileged documents included in the seed set. 18 19 identified and prioritized potentially responsive documents based on the initial training, Land 20
O'Lakes will review a sample set of those documents for responsiveness. The sample set will consist 21 of at least the 500 documents ranked most highly by Axcelerate in order of relevance. After this 22 second round of review, the system will be trained again based on reviewer feedback to identify and 23 prioritize more potentially responsive documents. Land O'Lakes will provide Plaintiffs with tracking 24 data for this round of review. 25
26 documents identified by Axcelerate as potentially responsive. During this review, the iterative 27
1 Axcelerate ranks returned documents in order of responsiveness to search terms.
3. Step Three -- Seed Set and Initial Training: All responsive documents identified
a) Approximately 400 documents already identified as responsive by Land O'Lakes;
4. Step Four -- Iterative Review and Further Training: Once Axcelerate has
5. Step Five -- Review: Land O'Lakes will then conduct a manual review of all process will continue; as more responsive documents are reviewed and coded, they will be used to 2 further hone the system's ability to identify and prioritize other potentially responsive documents 3 among the remaining, unreviewed portion of the document review corpus. Newly identified 4 documents also will be subject to manual review. Review and iterative training will proceed until all 5 identified documents have been reviewed and the system indicates that the remainder of the 6 document review corpus is not likely to be responsive. Land O'Lakes will continue to provide 7 tracking data to Plaintiffs, and all responsive, non-privileged documents will be produced. 8
9 iterative review and training to generate a conclusion from Axcelerate that the remaining documents 10 in the document review corpus are not likely to be responsive, Land O'Lakes will perform a 11 validation test by reviewing a statistically valid and random sampling of unreviewed documents to 12 confirm that the number of potentially responsive documents in the unreviewed corpus is statistically 13 insignificant. Using a 99% confidence level and a 1% estimation interval, a random sampling of the 14 unreviewed documents will be reviewed for responsiveness. If the number of responsive documents 15 in the validation sampling results in higher than a 1% responsiveness rate, the review process will 16 continue until the validation test achieves a 1% or less responsiveness rate. All tracking data 17 regarding the validation process will be provided to the plaintiffs, and any responsive, non-privileged 18 documents identified during the process will be produced. 19
THE PARTIES HEREBY STIPULATE AND AGREE TO THE ABOVE TERMS.
6. Step Six -- Validation: After Land O'Lakes has conducted enough rounds of
DATED: April 16, 2013 Respectfully submitted, 21 22 Matthew S. Kahn (SBN 261679) GIBSON, DUNN & CRUTCHER LLP 555 Mission Street, Suite 3000 San Francisco, CA 94105-2933 Telephone: (415) 393-8200 Fax: (415) 393-8200 email@example.com firstname.lastname@example.org By:/s/ Nathan P. Eimer Nathan P. Eimer (pro hac vice) Vanessa G. Jacobsen (pro hac vice) By:/s/ Matthew S. Kahn George A. Nicoud (SBN 106111)
Daniel D. Birk (pro hac vice) Sarah E. Malkerson (pro hac vice) EIMER STAHL LLP 224 South Michigan Avenue, Suite 1100 Chicago, Illinois 60604 Phone: (312) 660-7601 Fax: (312) 692-1718 email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org Attorneys for Defendant Land O' Lakes, Inc. By: /s/ Elaine T. Byszewski Elaine T. Byszewski HAGENS BERMAN SOBOL SHAPIRO LLP 301 North Lake Avenue, Suite 203 Pasadena, CA 91101 Telephone (213) 330-7150 Facsimile (213) 330-7152 email@example.com Steve W. Berman George W. Sampson Craig R. Spiegel HAGENS BERMAN SOBOL SHAPIRO LLP 1918 Eighth Avenue, Suite 3300 Seattle, WA 98101 Telephone (206) 623-7292 Facsimile (206) 623-0594 firstname.lastname@example.org email@example.com firstname.lastname@example.org Attorneys for the Plaintiffs ATTESTATION: I, Matthew S. Kahn, attest that concurrence in the filing of this document has been 21 obtained from each of the other signatories. 22 23
PURSUANT TO STIPULATION, IT IS SO ORDERED.
Appendix A: Search Terms
Land O'Lakes' Search Terms
"cooperatives working together" nmpf OR "national milk producers federation" 4 herd /10 retir* cow /10 retir* 5 cwt AND retir* "self help" 6 "herd buyout" "milk diversion program" 7 cwt AND beef cwt /5 program 8 export /10 assistance cwt /5 ambassador 9 cwt AND (independent /5 producer) cwt AND (independent /5 farmer) 10 cwt AND (target /10 remov*) cwt AND cull* 11 cwt AND slaughter* cwt AND "field staff" 12 cwt AND assessment cwt AND delegate 13 cwt AND (milk /5 suppl*) cwt AND (price /10 increas*) 14 cwt AND contribut* cooperative /10 individual 15 cwt AND (cow /5 numbers) cwt AND "farm id" 16 cents /5 hundredweight amount /10 checks 17 cwt AND bid* cwt AND "leadership council" 18 cwt /10 member* cwt /10 summary 19 cwt AND presentation 20 Plaintiffs' Search Terms buyout re-entry 22 re-entry re-enter 23 re-enter rejoin 24 re-join join /10 again 25 price /10 increase price /10 rise 26 price /10 raise price /10 hike 27 price /10 stabiliz* price /10 maintain 28 price /10 maintenance
© 1992-2013 VersusLaw Inc.