Pembrolizumab +/- SD-101 in Hormone-Naïve Oligometastatic Prostate Cancer With RT and iADT

This randomized phase II trial studies how well androgen deprivation therapy, pembrolizumab, and stereotactic body radiation therapy with or without toll-like receptor 9 (TLR9) agonist SD-101 in treating participants with prostate cancer and cancer in all oligometastatic sites that has spread to other places in the body. Androgen can cause the growth of tumor cells. Androgen deprivation therapy, such as leuprolide acetate, prednisone, and abiraterone acetate may lessen the amount of androgen made by the body. Monoclonal antibodies, such as pembrolizumab, may interfere with the ability of tumor cells to grow and spread. Stereotactic body radiation therapy is a specialized radiation therapy that sends x-rays directly to the tumor using smaller doses over several days and may cause less damage to normal tissue. Colony-stimulating factors, such as TLR9 agonist SD-101, may increase the production of blood cells. It is not yet known whether giving androgen deprivation therapy, pembrolizumab, and stereotactic body radiation therapy with or without TLR9 agonist SD-101 may work better in treating participants with prostate cancer and cancer in all oligometastatic sites. PRIMARY OBJECTIVES COHORT 1: \-- To assess the safety associated with giving RT and pembrolizumab with or without intratumoral SD-101. COHORT 2: * To continue to assess the safety associated with giving RT and pembrolizumab with or without intratumoral SD-101. * To assess if the rate of PSA \< nadir + 2 ng/mL at 15 months in participants with non-castrate levels of testosterone is greater than the historical control in each study arm. SECONDARY OBJECTIVES COHORT 2 ONLY: * To determine the rate of testosterone-prostate specific antigen (PSA) uncoupling in each study arm in Cohort 2. Testosterone-PSA uncoupling is defined as PSA \< 50% baseline and \< 20ng/mL for at least 3 months after testosterone recovers to \>150 ng/dL. In participants with metastatic hormone sensitive prostate cancer off hormonal therapy, \>90% of participants are expected to have PSA increase to \> 50% baseline after 3 months of testosterone recovery. * To estimate time to clinical progression in each study arm in Cohort 2. * To estimate progression-free survival (PFS) in each study arm Cohort 2. EXPLORATORY OBJECTIVE * To assess peripheral and tumor-based biomarkers of response and resistance in both cohorts. * To define the treatment-induced effects on circulating immune cells in both cohorts. * To explore remodeling of circulating T cell repertoire in both cohorts. * To explore the concordance of Prostate-Specific Membrane Antigen (PSMA) - Positron Emission Tomography (PET) scanning with conventional imaging in oligometastatic prostate cancer participants in both cohorts.