Rituximab and Pembrolizumab With or Without Lenalidomide in Treating Patients With Relapsed Follicular Lymphoma and Diffuse Large B-Cell Lymphoma

PRIMARY OBJECTIVES: I. To determine the overall response rate (ORR) in subjects with relapsed follicular lymphoma (FL) treated with rituximab plus pembrolizumab. II. To determine the ORR in subjects with relapsed/refractory FL and relapsed/refractory diffuse large B-cell lymphoma (DLBCL) who have failed chimeric antigen receptor (CAR) T cell therapy and are treated with rituximab in combination with pembrolizumab and lenalidomide. (Cohort 2) SECONDARY OBJECTIVES: I. To determine the safety and toxicity. II. To determine the complete response rate (CRR). III. To determine the overall progression-free survival (PFS). IV. To compare PFS between patients relapsing =\< one year vs \> one year after last prior therapy. V. To determine the overall survival (OS). VI. To determine the safety and toxicity. (Cohort 2) VII. To determine the CRR. (Cohort 2) VIII. To determine the overall PFS. (Cohort 2) IX. To compare PFS between patients relapsing =\< one year vs \> one year after last prior therapy. (Cohort 2) X. To determine the OS. (Cohort 2) EXPLORATORY OBJECTIVES: I. To determine effects of rituximab plus pembrolizumab therapy on peripheral blood T cells. II. To correlate features of peripheral blood T cells with toxicities after rituximab plus pembrolizumab therapy. III. To correlate features of peripheral blood T cells with response and PFS after rituximab plus pembrolizumab therapy. IV. To correlate baseline tumor characteristics with response and PFS after rituximab plus pembrolizumab therapy. V. To determine effects of rituximab, pembrolizumab, and lenalidomide therapy on peripheral blood T cells. (Cohort 2) VI. To correlate features of peripheral blood T cells with toxicities after rituximab, pembrolizumab, and lenalidomide therapy. (Cohort 2) VII. To correlate features of peripheral blood T cells with response and PFS after rituximab, pembrolizumab, and lenalidomide therapy. (Cohort 2) VIII. To correlate baseline tumor characteristics with response and PFS after rituximab, pembrolizumab, and lenalidomide therapy. (Cohort 2) OUTLINE: Patients are assigned to 1 of 2 cohorts. COHORT I: Patients receive rituximab intravenously (IV) over 4-8 hours on days 1, 8, 15, and 22. Patients also receive pembrolizumab IV over 1 hour on day 2 every 3 weeks for up to 16 cycles (1 year) in the absence of disease progression or unacceptable toxicity. COHORT II: Patients receive rituximab IV over 4-8 hours on days 1, 8 and 15 of cycle 1, and day 1 of cycle 2. Patients also receive pembrolizumab IV over 1 hour on day 2 every 3 weeks for up to 2 years, and lenalidomide orally (PO) on days 1-14 every 3 weeks for up to 12 cycles in the absence of disease progression or unacceptable toxicity. After completion of study treatment, patients are followed up at 30 days, every 3 months for 1 year, and then every 6 months thereafter.