Skip to content

Commit 595cf68

Browse files
shivankgarg98akpm00
authored andcommitted
mm/khugepaged: fix race with folio split/free using temporary reference
hpage_collapse_scan_file() calls is_refcount_suitable(), which in turn calls folio_mapcount(). folio_mapcount() checks folio_test_large() before proceeding to folio_large_mapcount(), but there is a race window where the folio may get split/freed between these checks, triggering: VM_WARN_ON_FOLIO(!folio_test_large(folio), folio) Take a temporary reference to the folio in hpage_collapse_scan_file(). This stabilizes the folio during refcount check and prevents incorrect large folio detection due to concurrent split/free. Use helper folio_expected_ref_count() + 1 to compare with folio_ref_count() instead of using is_refcount_suitable(). Link: https://lkml.kernel.org/r/20250526182818.37978-1-shivankg@amd.com Fixes: 05c5323 ("mm: track mapcount of large folios in single value") Signed-off-by: Shivank Garg <shivankg@amd.com> Reported-by: syzbot+2b99589e33edbe9475ca@syzkaller.appspotmail.com Closes: https://lore.kernel.org/all/6828470d.a70a0220.38f255.000c.GAE@google.com Suggested-by: David Hildenbrand <david@redhat.com> Acked-by: David Hildenbrand <david@redhat.com> Acked-by: Dev Jain <dev.jain@arm.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Cc: Bharata B Rao <bharata@amd.com> Cc: Fengwei Yin <fengwei.yin@intel.com> Cc: Liam Howlett <liam.howlett@oracle.com> Cc: Lorenzo Stoakes <lorenzo.stoakes@oracle.com> Cc: Mariano Pache <npache@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com> Cc: Zi Yan <ziy@nvidia.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
1 parent e13e792 commit 595cf68

1 file changed

Lines changed: 17 additions & 1 deletion

File tree

mm/khugepaged.c

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2293,6 +2293,17 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,
22932293
continue;
22942294
}
22952295

2296+
if (!folio_try_get(folio)) {
2297+
xas_reset(&xas);
2298+
continue;
2299+
}
2300+
2301+
if (unlikely(folio != xas_reload(&xas))) {
2302+
folio_put(folio);
2303+
xas_reset(&xas);
2304+
continue;
2305+
}
2306+
22962307
if (folio_order(folio) == HPAGE_PMD_ORDER &&
22972308
folio->index == start) {
22982309
/* Maybe PMD-mapped */
@@ -2303,23 +2314,27 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,
23032314
* it's safe to skip LRU and refcount checks before
23042315
* returning.
23052316
*/
2317+
folio_put(folio);
23062318
break;
23072319
}
23082320

23092321
node = folio_nid(folio);
23102322
if (hpage_collapse_scan_abort(node, cc)) {
23112323
result = SCAN_SCAN_ABORT;
2324+
folio_put(folio);
23122325
break;
23132326
}
23142327
cc->node_load[node]++;
23152328

23162329
if (!folio_test_lru(folio)) {
23172330
result = SCAN_PAGE_LRU;
2331+
folio_put(folio);
23182332
break;
23192333
}
23202334

2321-
if (!is_refcount_suitable(folio)) {
2335+
if (folio_expected_ref_count(folio) + 1 != folio_ref_count(folio)) {
23222336
result = SCAN_PAGE_COUNT;
2337+
folio_put(folio);
23232338
break;
23242339
}
23252340

@@ -2331,6 +2346,7 @@ static int hpage_collapse_scan_file(struct mm_struct *mm, unsigned long addr,
23312346
*/
23322347

23332348
present += folio_nr_pages(folio);
2349+
folio_put(folio);
23342350

23352351
if (need_resched()) {
23362352
xas_pause(&xas);

0 commit comments

Comments
 (0)