The baseline point generalises past ML. “We improved it 30%” — over what? Half our internal wins are noise above an unmeasured control.
Grace Lin@gracelin· Jun 15
Please report your baseline. A model that’s 92% accurate means nothing until I know that guessing the majority class gets 90%. Half the “breakthroughs” I review are two points above a coin flip nobody measured.
%20by%20%E2%80%9EZoish%E2%80%9D%2C%20licensed%20under%20%E2%80%9ECC0%201.0%E2%80%9D%20(https%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F)%3C%2Fdc%3Arights%3E%3C%2Frdf%3ADescription%3E%3C%2Frdf%3ARDF%3E%3C%2Fmetadata%3E%3Cmask%20id%3D%22viewboxMask%22%3E%3Crect%20width%3D%22560%22%20height%3D%22560%22%20rx%3D%22280%22%20ry%3D%22280%22%20x%3D%220%22%20y%3D%220%22%20fill%3D%22%23fff%22%20%2F%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23viewboxMask)%22%3E%3Crect%20fill%3D%22%23d1d4f9%22%20width%3D%22560%22%20height%3D%22560%22%20x%3D%220%22%20y%3D%220%22%20%2F%3E%3Cg%20transform%3D%22translate(136%20328)%22%3E%3Cpath%20d%3D%22M175%2075.5c8.2-5%2011.5-.1%2014.5%205.4-11.4%2017-50%2020.1-60.7%203.4%2015-5.2%2031.3-1.5%2046.1-8.8Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(246%20125)%22%3E%3Cpath%20d%3D%22M65.7%2043.5c-4.3%2023.6-7.3%2045.8-.3%2068.8A79.4%2079.4%200%200%200%2098%20157.1c22.3%2014.8%2022.7%2043%20.4%2057.5-24.6%2016.2-50.8%2014-77%207.5-11.8-3-17.9-14.2-21-25.4-2.7-9.6%206.5-25.2%2016.6-31%203.1%203.8%202.4%207%20.2%2011.3-6.7%2012.3%200%2031.4%2013.5%2032.6%2019.8%201.7%2040.7%205.8%2059.1-8%2012.5-9.4%2014-21.5%201.8-31.4-7.3-5.9-14.4-11.7-20.6-18.8-28-31.7-29.9-68.2-20-107%203.5-14.3%209.2-27%2019-38C73%203%2076-.9%2080.5%202c5.9%203.9%201.2%208.4-1%2012-5.6%209-10.6%2018.3-13.8%2029.5Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(-45%20137)%22%3E%3Cpath%20d%3D%22M419.6%20111c-4.2-9.5-9.9-15.2-19.8-15.7-6-.3-7-5.1-8.9-9.8%202-1.5%203.5-3.7%205.3-3.8%2021.1-1.7%2042.4-1.8%2063.5-.1%206%20.4%209.3%208%203.2%2010.4-10%204-12.5%2013.2-18.5%2019.8-8.2%209-15.2%209.1-24.8-.8ZM228.6%2081.4c12.2.7%2023.1%201%2034%202%203.6.3%208%201.8%208%206.7%200%205.6-4.6%205.2-8.3%205.1-6.3%200-9.7%202.7-12.5%208.6-3.4%207.4-6.9%2015.3-18%2014.8-10.5-.5-15.1-7.2-18.1-15.2-2.2-5.7-5-8.4-11-8.2-3.4.1-7.1-.2-7.5-4.9-.4-5%203-6%207-7.2a67.9%2067.9%200%200%201%2026.4-1.7Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(-45%20137)%22%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(119%20114)%22%3E%3Cpath%20d%3D%22M295%2029.6c2.6%207.3%205.5%2013.5%207%2020%201.2%204.9%201.7%2010.4-5%2012.6-7.6%202.6-8.6-3.4-10.6-7.8a120%20120%200%200%201-5.5-14.2c-1.5-4.9-4.3-7-9.4-6.8-5%20.1-8%203-9.3%207.2-1.7%205.3-3%2010.7-4%2016.1-1%204.5-4.2%204.8-7.8%204.6-3.6-.1-5.3-2-5.9-5.5-3.2-20.1%206.1-38.2%2025.9-41.2%2012.2-2%2019%204.5%2024.5%2015ZM50.6%2033C65.7%2012.5%2080%208.5%2094.6%2021.7c9.1%208.3%208.7%2021.2%2010.7%2032.6.5%203-2%205.7-5.3%206.3-3.5.7-7-1.2-8.4-4.4-1.7-4-2.2-8.3-3.6-12.2-1.7-4.8-3.2-10-9.7-10.5-6.4-.5-10.6%203-13.2%208-3.6%207.2-4.8%2020-15.7%2014.7-10.4-5-1.9-14.7%201.2-23.2Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3C%2Fg%3E%3C%2Fsvg%3E)
%20by%20%E2%80%9EZoish%E2%80%9D%2C%20licensed%20under%20%E2%80%9ECC0%201.0%E2%80%9D%20(https%3A%2F%2Fcreativecommons.org%2Fpublicdomain%2Fzero%2F1.0%2F)%3C%2Fdc%3Arights%3E%3C%2Frdf%3ADescription%3E%3C%2Frdf%3ARDF%3E%3C%2Fmetadata%3E%3Cmask%20id%3D%22viewboxMask%22%3E%3Crect%20width%3D%22560%22%20height%3D%22560%22%20rx%3D%22280%22%20ry%3D%22280%22%20x%3D%220%22%20y%3D%220%22%20fill%3D%22%23fff%22%20%2F%3E%3C%2Fmask%3E%3Cg%20mask%3D%22url(%23viewboxMask)%22%3E%3Crect%20fill%3D%22%23c0aede%22%20width%3D%22560%22%20height%3D%22560%22%20x%3D%220%22%20y%3D%220%22%20%2F%3E%3Cg%20transform%3D%22translate(136%20328)%22%3E%3Cpath%20d%3D%22M185.6%2059.2c7.2%208.7%209.1%2020%2020.2%2025%204.5%202%202.4%207.7-2.9%209.1a31%2031%200%200%201-10.1.5c-22.4-1.6-44.7-3.4-67.2.5-7.7%201.3-16-.3-24-.7-4.3-.2-7.5-2.4-8-6.8s3.5-6.4%206.5-6.5c13.7-.4%2021-10.2%2028.8-19%206.8-8%2015-9.4%2024.8-9.5%2011.1-.2%2022.7-4%2032%207.4ZM170.2%20119.5c-10%201.2-19.5%203.4-28-6.2%209.4-4.8%2017.9-4.2%2026-4.8%203-.2%206.9%201.9%207.6%205.7.5%202.9-2.3%204.1-5.6%205.3ZM107.8%2057.2c.3%204-5%206.8-8.2%204.4a5.4%205.4%200%200%201-2.2-4.4c0-3.9%203.8-5.1%207.2-5.7%201.3-.2%202.9%201.9%203.2%205.7Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(246%20125)%22%3E%3Cpath%20d%3D%22M69.4%2096c-.6-14.1-4.3-27.2.4-40.1%201.2-3.3%201.2-8.6%206.7-8%205.5.8%204.5%205.6%204%209.2-4.6%2030.7%207%2058.6%2015%2087%202.7%209.3%206.6%2018.2%206.1%2028.4-.8%2016.4-11.3%2026.4-27.4%2023.6a29.5%2029.5%200%200%200-17.2%201.8c-14%205.6-26-5-39.6-5.3-4%200-6.6-7.3-7-12.7%200-2.5.4-5.2%203.5-6%202-.8%204.4.2%205.3%202.2%203.6%207%209.3%203.2%2014.4%204.2%2011.9%202%2023%201.3%2034.8.9%205.8-.3%2011.7%203.3%2017.2-1.7%205.8-5.2%204-11.3%202.2-17.6C81.7%20140.3%2072%20119.7%2069.4%2096Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(-45%20137)%22%3E%3Cpath%20d%3D%22M409.2%2075.7c13.1.3%2025.4%201%2037.7%202.5%203%20.3%204.9%203.3%205%206.6%200%203.4-1.8%205.7-4.8%207-11.1%204.9-22%208.3-33.7.2-6.5-4.4-8.9-8.6-4.2-16.3ZM254.7%2078.4c6.2%203.3%205.2%207%202.1%2010.8-6.5%208-28.2%2010-36.4%203.5-4.7-3.8-7.3-8.6-4.4-14.4%203-6%208.2-5%2013-2.5%208%204%2016.4%203%2025.7%202.6Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(-45%20137)%22%3E%3C%2Fg%3E%3Cg%20transform%3D%22translate(119%20114)%22%3E%3Cpath%20d%3D%22M53.5%2037.6a79.2%2079.2%200%200%200%2027.3-19.8c2.7-3%206.1-5.7%209.9-5%206.7%201.1%2016.2-1%2018.8%207.5%202.1%207-5.2%2010.5-10.3%2014.4-13.8%2010.5-28.2%2019-46.6%2016.7-4.6-.5-9.7.2-11-6.2%201.6-5.5%207.3-4.9%2011.9-7.6ZM251.4%2015c8.5-1.7%2012.6%204.1%2017.1%208.2%208.1%207.4%2014.7%2016.4%2026%2019.8%203.3%201%205.7%204%204%208-1.5%203.5-4%205.7-8%204.6-14.3-3.8-29.2-5.9-40.6-16.8-10-9.6-10.2-14%201.5-23.7Z%22%20fill%3D%22%23000%22%2F%3E%3C%2Fg%3E%3C%2Fg%3E%3C%2Fsvg%3E)