UNITED STATES DISTRICT COURT SOUTHERN DISTRICT OF CALIFORNIA
January 12, 2010
PHUONG DOAN, ET AL., PLAINTIFF,
MICHAEL J. ASTRUE, COMMISSIONER OF SOCIAL SECURITY ADMINISTRATION, DEFENDANT.
The opinion of the court was delivered by: Hon. Dana M. Sabraw United States District Judge
ORDER: (1) DENYING DEFENDANT'S EX PARTE MOTION TO STRIKE; (2) GRANTING IN PART AND DENYING IN PART DEFENDANT'S MOTION TO EXCLUDE PLAINTIFFS' EXPERT; AND (3) DENYING PLAINTIFFS' MOTION TO EXCLUDE DEFENDANT'S EXPERT [Docs. 290, 301, 302]
Pending before the Court are: (1) Defendant's motion to exclude Plaintiffs' statistician expert, Dr. James Lackritz, (2) Defendant's ex parte motion to strike Dr. Lackritz' third report and declaration, and (3) Plaintiffs' motion to exclude Defendant's expert witness, Dr. Benjamin Shippen. For the reasons set forth below, Defendant's ex parte motion is denied, Defendant's motion to exclude Plaintiffs' expert is granted in part and denied in part, and Plaintiffs' motion to exclude Defendant's expert is denied.
This case arises out of Plaintiffs' claims that Steven New, a former Administrative Law Judge for the Social Security Administration ("ALJ New"), was biased against social security claimants. On August 21, 2006, the matter was certified as a class action, with the class defined as "all claimants for Title II Social Security Disability Insurance Benefits or Title XIV Supplemental Security Income whose claims had been assigned to ALJ New and have had their claims denied or dismissed by ALJ New, with the exception of claimants whose claims are time barred under 42 U.S.C. § 405(g)." (Doc. 65, Order Granting Class Cert. at 9-10.)
Plaintiffs retained James R. Lackritz, Ph.D., to analyze certain data and provide statistical evidence of ALJ New's bias. Dr. Lackritz made the following conclusions: (1) that ALJ New handled significantly fewer cases than national and San Diego norms; and (2) that ALJ New allowed significantly fewer claims than national norms. (Schweinner Dec., Ex. 2 at 5-6.)
Defendant sought an expert opinion from Dr. Benjamin S. Shippen, Jr. Dr. Shippen made the following conclusions: (1) Dr. Lackritz' productivity opinion contains flaws in both data and analysis; (2) Dr. Lackritz' allowance opinion is flawed because ALJ New should have been compared to other California ALJs, rather than to a national benchmark; (3) Dr. Lackritz used an inappropriate statistical test to arrive at his allowance opinion; and (4) ALJ New's allowance rate is not statistically significant when compared to California ALJs. (Manbeck Dec., Ex. B at 13.)
At issue are three reports prepared by Dr. Lackritz, and two reports prepared by Dr. Shippen. Dr. Lackritz' initial report ("Lackritz 1"), dated August 12, 2009, compared ALJ New's calendar year figures against fiscal year nationwide data to arrive at a productivity opinion. (Schweinner Dec., Ex. 1; Lackritz Dep. 78:6-16.) Dr. Lackritz' second report ("Lackritz 2") revises the analysis to reflect fiscal year data. (Schweinner Dec., Ex. 2.) Lackritz 2 was provided to Defendant at Dr. Lackritz' deposition on September 22, 2009. During the deposition, Dr. Lackritz discovered that figures he used to arrive at his allowance opinion were incorrect. (Lackritz Dep., 72:9-74:17.) He revised those figures and provided an amended report following his deposition ("Lackritz 3"). (Manbeck Dec., Ex. P.) His conclusions are the same in each report.
Dr. Shippen's initial report ("Shippen 1") is dated September 20, 2009. (Manbeck Dec., Ex. A). Following Dr. Lackritz' deposition and second report, Dr. Shippen provided a revised report, dated October 5, 2009 ("Shippen 2"), which reached the same conclusions as his initial opinion. (Manbeck Dec., Ex. B.)
On October 9, 2009, Defendant filed a motion to exclude Dr. Lackritz' expert opinion under Rule 702 of the Federal Rules of Evidence. (Doc. 290). Plaintiffs filed an opposition, and Defendant filed a reply.*fn1 (Docs. 298 & 300). Defendant also filed an ex parte motion to strike evidence that Plaintiffs submitted with their opposition. (Doc. 301.) These motions are addressed below.
A. Motion to Strike
Plaintiffs filed Lackritz 3 and a declaration of Dr. Lackritz with their opposition to Defendant's motion to exclude. Defendant moves to strike this evidence as untimely and in violation of an order issued by Magistrate Judge Brooks.
Lackritz 1 originally was provided after the deadline set in the Rule 16 scheduling order. Because of this discovery violation, Defendant filed a motion with the Magistrate Judge to exclude Plaintiffs' expert witness testimony. (Doc. 217.) Magistrate Judge Brooks declined to exclude Dr. Lackritz' testimony, but precluded Plaintiffs from supplementing the report. (Doc. 255, September 1, 2009 Order, p. 4.) Nevertheless, Plaintiffs submitted Lackritz 2, which is now the subject of Defendant's motion to exclude.
During Dr. Lackritz' deposition, he discovered an error in his report. In reaching his conclusion that ALJ New allowed significantly fewer claims than national norms, Dr. Lackritz utilized figures from a report entitled Key Workload Indicators, Hearings-Appeals-Civil Actions-Attorney Fees, Second Quarter Fiscal Year 2008 ("KWI"). (Schweinner Dec., Ex. 7.) That report provides the national disposition rates of unfavorable, favorable, partially favorable, and dismissed cases for fiscal years 1999-2007. (KWI at 4). The chart column for favorable rates includes both favorable and partially favorable rates. Partially favorable rates are also listed separately. Thus, when adding the percentages of unfavorable, favorable, partially favorable, and dismissed decisions, the total is more than 100% because the partially favorable rates are included twice. Dr. Lackritz, not realizing that the partially favorable numbers were twice included, adjusted the figures in his report to account for the total being more than 100%. (Lackritz Dep. 72:9-73:24.)
Dr. Lackritz therefore offered to provide a new report at his expense. (Lackritz Dep. 74:14-17.) He stated that the mistake did not change his opinion, only that it changed his numbers slightly. (Id.) Lackritz 3 represents Dr. Lackritz' updated report. The only differences between Lackritz 2 and Lackritz 3 are the chart on page 4, which lists the national favorable rates, and the chart on page 5, which lists the results of a standard deviation calculation based on the national rates. Although this report was created after Dr. Lackritz' deposition, it is clear from the deposition testimony that the report merely corrects an inadvertent error. The new report does not provide any new opinions and is not based on new data. The Court therefore denies Defendant's motion to strike Lackritz 3, subject to Dr. Lackritz being re-deposed at Defendant's discretion and at Plaintiffs' expense.
Defendant also seeks to strike Dr. Lackritz' declaration of October 30, 3009, which refutes several statements in Dr. Shippen's expert report. (Doc. 303-2.) Defendant argues that the declaration was prepared solely to avoid exclusion of Dr. Lackritz' opinion testimony, and that there has been no opportunity to depose Dr. Lackritz on the new information. Plaintiff argues the declaration is entirely consistent with Dr. Lackritz' expert report and deposition testimony.
At the time of Dr. Lackritz' deposition, he only had one day to review Dr. Shippen's report. (Lackritz Dep. 7:20-8:3.) Dr. Lackritz indicated that his deposition testimony covered his significant objections to Dr. Shippen's report, but that he might have more objections if he had time to review the report. (Lackritz Dep. 100:15-25.) Defense counsel indicated she would be continuing the deposition and this would give him additional time to review the report. (Id. at 101:3-15.) Defendant did not depose Dr. Lackritz a second time and thus, Dr. Lackritz did not have an opportunity to object in more detail to Dr. Shippen's opinions. Under these circumstances, the Court denies Defendant's motion to strike the declaration, but permits Defendant to re-depose Dr. Lackritz at Plaintiffs' expense on the statements made in the declaration.
B. Defendant's Motion to Exclude Dr. Lackritz' Opinions
Federal Rule of Evidence 702 provides:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training or education, may testify thereto in the form of an opinion or otherwise, if (1) the testimony is based upon sufficient facts or data, (2) the testimony is the product of reliable principles and methods, and (3) the witness has applied the principles and methods reliably to the facts of the case.
Fed. R. Evid. 702. Under this Rule, expert testimony must be both reliable and relevant. Daubert v. Merrell Dow Pharms., Inc., 509 U.S. 579, 589 (1993).Defendant seeks to exclude Dr. Lackritz' opinions for being both unreliable and irrelevant.
1. Productivity Opinion
Dr. Lackritz' first conclusion is that ALJ New handled significantly fewer cases than national and San Diego norms during the years 1998-2002. (Lackritz 3 at 1-3, 5.) Dr. Lackritz reaches this conclusion using a statistical analysis that compares the number of cases handled by ALJ New to the national and San Diego norms during the relevant time period. (Id. at 2.) However, Dr. Lackritz states in his report that he did not have national and San Diego norms for the relevant years. (Id.) Thus, he created a national norm of 400 cases and a San Diego norm of 340 cases. (Id.)
During his deposition, Dr. Lackritz explained his methodology for creating the national and San Diego norms. (Lackritz Dep., 62:13-64:14; 104:4-11.) He took data from a newspaper article that provided the average number of case dispositions per administrative law judge for the years 2005-2007. In 2005, the national average was 421, while the San Diego average was 359. In 2006, the national average was 459, while San Diego was 438. In 2007, the national average was 474, while San Diego was 344. (Schweiner Decl., Ex. 8.) Dr. Lackritz then made his "best estimate" as to what the averages may have been for 1998-2002. (Lackritz Dep., 62:18-25.) Since the national average was well over 400 for the years 2005-2007, Dr. Lackritz assumed 400 was a conservative estimate for 1998-2002. (Id. at 61:1-6.) Dr. Lackritz explained that "what [he] was trying to do was create some figure to test against Judge New, which [he] thought would be appropriate and a judge or jury looking at this particular case could say, 'That's reasonable.' The 400 in all these three years, they're well over 400, so I think 400 is a reasonable number." (Id. at 63:13-18.)
Defendant argues this opinion is unreliable because it is a guess not supported by actual data. Plaintiffs argue the opinion is reliable because Dr. Lackritz was merely extrapolating data from one period to another. Plaintiffs define extrapolation as "the application of knowledge about an event to a series of similar events, using 'inferences based on an assumed continuity, correspondence, or other parallelism.'" The problem with Plaintiffs' argument is that Dr. Lackritz did not make any inferences based on assumed continuity, correspondence, or other parallelism. Dr. Lackritz did not engage in an analysis, he simply reached the benchmark figures "subjectively after examining the data from the newspaper." (Id. at 64:6-8.) Dr. Lackritz did not mention or try to account for the myriad factors that may affect productivity rates nationwide. Rather, Dr. Lackritz chose his national benchmark because he thought "400 [was] a reasonable number." (Id. at 62:4-18.)
Further, while expert testimony may be based on hearsay, the hearsay must be reliable. Here, the 2005-2007 productivity numbers used by Dr. Lackritz came from a newspaper article, not from a primary source. The newspaper article states that the numbers are from a report by the Social Security inspector general. (Schweiner Decl., Ex. 8.) However, there is no mention that Dr. Lackritz ever viewed this source and verified the data. Thus, the Court finds that the productivity opinion, based on numbers Dr. Lackritz created without supporting data, is unreliable.
Dr. Lackritz' opinion is also irrelevant. The issue at hand is whether ALJ New was biased toward Social Security claimants. The number of cases he decided in a year sheds no light on this issue. Plaintiffs argue it is relevant because ALJ New was undergoing cancer treatment and this affected his judicial demeanor. Plaintiffs attempt to bolster their argument with a declaration from attorney David Shore, who appeared in front of ALJ New and recalls a change in his judicial temperament toward the end of his tenure. Mr. Shore, however, states that he does not recall any bias on the part of ALJ New against claimants. Further, Plaintiff makes no connection between a lack of productivity and bias. Accordingly, the opinion is excluded.
2. Allowance Rates
Dr. Lackritz' second opinion is that from 1997 through 2003, ALJ New allowed significantly fewer claims than national norms. Using a statistical calculation of p-value and standard deviation, Dr. Lackritz concluded that ALJ New's low allowance rates were statistically significant. Dr. Lackritz concluded that the low numbers were not due to random chance and that "the number of allowed decisions rendered by ALJ New is not even close to what should be the worst case scenario for the national distribution." (Lackritz 3 at 3-6.)
Defendant first argues that the opinions regarding the years 1997 and 2003 should be excluded because ALJ New did not work the entire year, yet his allowance rates were compared against full-year national rates. Plaintiffs do not specifically respond to this argument, but instead state that even if those two years are excluded, ALJ New's allowance rates remain significantly low. Given that ALJ New worked only three months in 1997 and seven months in 2003, the Court finds that Plaintiffs' analysis is unreliable as it compares ALJ New's decision rates to national figures based on a twelve month cycle. Thus, the Court strikes the opinion testimony of Dr. Lackritz as to the years1997 and 2003.
Defendant next argues the analysis for 1997 and 1998 is flawed because Dr. Lackritz did not have any nationwide data for those years. Dr. Lackritz relied on two reports to establish the nationwide figures. The KWI report, discussed above, gives the nationwide figures for 1999-2007. For 1997 and 1998, Dr. Lackritz used a May 2006 report from the Social Security Advisory Board, Disability Decision Making: Data and Materials ("DDM"). The report does not provide specific figures for nationwide allowance rates for 1997 and 1998. Rather, a line graph shows the pattern of allowance rates from 1990 through 2004. From the line graph, Dr. Lackritz estimated that the national allowance rate was 64% in 1997 and 65% in 1998. (Lackritz 3 at 4.)
Plaintiff argues the figures are reliable because nationwide statistics were provided in another case, Pronti v. Barnhart, 339 F. Supp. 2d 480 (W.D.N.Y. 2004), where the allowance rates were 65.1% in 1997 and 62.6% in 1998. Plaintiff argues the figures are similar and, therefore, Dr. Lackritz' opinions are reliable. In statistical analysis, however, a change in the underlying data may change the calculations. This is evidenced by Dr. Lackritz' need to create Lackritz 3 after he discovered the mistake in his calculations. Although he stated the differences did not change his opinions, he noted that the differences would change his numbers slightly. (Lackritz Dep., 74:14-17.) Here, a 2.4% difference exists between Dr. Lackritz' estimated figure for 1998 and the figure used in Pronti. This change clearly would affect Dr. Lackritz' calculations of p-value and standard deviation. It may not change his ultimate opinion, but it renders his analysis for 1997 and 1998 unreliable.
Lastly, Defendant argues that Dr. Lackritz' analysis is flawed for the remaining years because Dr. Lackritz should have compared ALJ New's allowance rates to those of other California judges, rather than using a nationwide comparison. According to the DDM, there are many factors that may affect the consistency of disability decision making. (Schweinner Dec., Exh. 6, p. 7-8.) These factors include economic and demographic differences among states, as well as differences in administrative practices among different hearing offices. (Id.) Defendant's expert opines that because of these differences, it is inappropriate to compare a single ALJ to a nationwide benchmark. (Shippen 2 at 4.) Dr. Shippen opines that when comparing ALJ New's allowance rate to a California rate, the results are not statistically significant. (Id. at 9-13.)
Plaintiffs argue that comparing ALJ New's allowance rates to nationwide figures is appropriate because the methodology was approved in Pronti v. Barnhart, 339 F. Supp. 2d 480. In that case, which also involved an ALJ accused of bias, the court stated that attorney affidavits regarding the ALJ's bias, as well as statistical evidence that the ALJ denied claims at a higher than average rate, warranted an investigation by the SSA. Id. at 492-494. Contrary to Plaintiffs' argument, however, the court did not approve a particular methodology. The court was not ruling on the admissibility of expert testimony. Rather, the statistics were provided in raw form, without expert opinion as to their meaning. The court did not address whether a nationwide or regional comparison was appropriate. Id. Thus, Pronti is not determinative of the reliability of Dr. Lackritz' opinion.
Plaintiffs fail to explain how a nationwide benchmark would be a better or more accurate comparison than a California benchmark.*fn2 Nevertheless, the Court is not persuaded that Dr. Lackritz' methodology is unreliable. His use of a nationwide comparison pool ultimately goes to the weight of his opinion, not its reliability. Thus, the Court declines to find that the opinion testimony of Dr. Lackritz is unreliable regarding the years 1999 through 2002.
Defendant next argues that Dr. Lackritz' opinion is irrelevant because it does not address the cause of alleged bias or that lower allowance rates could be caused by factors unrelated to bias. Plaintiffs do not specifically respond to this argument, other than to say that the court in Pronti found the statistical evidence relevant.*fn3
Statistical evidence is often used in employment discrimination cases to help a plaintiff establish a prima facie case of discrimination. Hemmings v. Tidyman's Inc., 285 F.3d 1174, 1184 (9th Cir. 2002). Statistical analysis need not account for all variables. Bazemore v. Friday, 478 U.S. 385, 400 (1986). Failure to include relevant variables affects the weight of the evidence, not the admissibility. Id.; Obrey v. Johnson, 400 F.3d 691, 695 (9th Cir. 2005) ("[O]bjections to a study's completeness generally go to the 'weight, not the admissibility of the statistical evidence.'").
Here, Dr. Lackritz states that he does not know why discrepancies exist and that he renders no opinion on the cause of ALJ New's approval rates or whether ALJ New was biased. (Lackritz 2 at 6; Lackritz Dep., 98:1-100:9.) Further, Dr. Lackritz does not account for independent variables. His analysis simply compares ALJ New's allowance rates to those of other judges, coupled with his opinion that the lower rate is statistically significant, i.e., that it is not due to "random chance."*fn4 Presumably, Dr. Lackritz' opinion is just one piece of Plaintiffs' evidence of bias. See Obrey, 400 F.3d at 696-97 (noting that statistical disparity evidence alone could not prove discrimination but that it "should have been admitted for whatever probative value it had."). Thus, the Court finds that the evidence is relevant.
Accordingly, the Court grants Defendant's motion to exclude Dr. Lackritz' opinions regarding (a) the productivity of ALJ New, and (b) the allowance rates of ALJ New for 1997, 1998, and 2003. Defendant's motion is denied as to Dr. Lackritz' opinion regarding ALJ New's allowance rates for the years 1999 through 2002.
C. Plaintiffs' Motion to Exclude Dr. Shippen's Opinions
Plaintiffs move to exclude Defendant's expert opinions on the grounds that the underlying data is unreliable under Rule 702. Plaintiffs also argue the report violates Rule 26(a)(2)(B) of the Federal Rules of Civil Procedure.
Much of the data underlying Dr. Shippen's expert reports is from a spreadsheet which provides workload figures and favorable/unfavorable decision rates for ALJs in California during the relevant time period. (Doc. 302-10.) The spreadsheet was created by Defendant for purposes of this litigation. Each ALJ in California was assigned an anonymous letter code. (Sapp. Dep. 36:2-16.) ALJ New was given the code "DW."
Initially, Plaintiffs argue that because the identities of the ALJs were not disclosed, Dr. Shippen's report does not adequately explain the basis for its opinions and therefore the report violates Rule 26(a)(2)(B). That rule requires that experts provide a written report which contains, in part: "(i) a complete statement of all opinions the witness will express and the basis and reasons for them; (ii) the data or other information considered by the witness in forming them; (iii) any exhibits that will be used to summarize or support them..." Id.
Here, Dr. Shippen's report states that he relied on the spreadsheet containing California data. That spreadsheet was provided to Plaintiffs. Rule 26 does not require that the identities of the individual ALJs in California be disclosed. In fact, the nationwide data used by Dr. Lackritz is similarly anonymous. Moreover, while Plaintiffs complain that Defendant never produced the identities of the ALJs, Plaintiffs only made such a request after the deadline for written discovery had passed. Additionally, Plaintiffs requested only the identities of San Diego ALJs, which is not the comparison pool used by Dr. Shippen in his analysis. (Manbeck Dec., Exs. E-1 & G.)
Plaintiffs also argue that the data in the excel spreadsheet is unreliable, rendering Dr. Shippen's opinions inadmissible under Rule 702. Plaintiffs argue that the data in the spreadsheet is tainted by the inclusion of an ALJ who previously was found to be biased, that the data is unreliable because it was created ad hoc for this litigation, and that California ALJs are not the appropriate comparison pool. Plaintiffs' arguments are unavailing. As an initial matter, the ALJ referred to in Plaintiffs' moving papers was never found by a court to be biased. In Dut Le v. Commissioner, 98-cv-1896 (S.D. Cal. 2001), an ALJ in San Diego was accused of bias, but the case did not end in a judgment or finding of bias as the case settled. Thus, the inclusion of this ALJ in the spreadsheet does not taint the data pool. Similarly, the fact that the spreadsheet was created for this litigation does not render the data unreliable. There is no assertion that the actual figures provided in the spreadsheet are incorrect. In fact, Plaintiffs' expert also relied on the spreadsheet in order to obtain the productivity and allowance rates of ALJ New. (Lackritz 3 at 1.) Finally, the fact that Dr. Shippen used a California comparison pool, rather than a nationwide or San Diego pool, affects the weight of the opinion, not its reliability. Accordingly, Plaintiffs' motion to exclude Defendant's expert is denied.
For the reasons stated above, Defendant's ex parte motion to strike is denied, Defendant's motion to exclude Plaintiffs' expert is granted in part and denied in part, and Plaintiffs' motion to exclude Defendant's expert is denied.
IT IS SO ORDERED.