Navitoclax and Sorafenib Tosylate in Treating Patients With Relapsed or Refractory Solid Tumors

Inclusion Criteria

• •For Dose Escalation Cohort: Patients must have histologically confirmed malignancy that is metastatic or unresectable and for which standard curative or palliative measures do not exist or are no longer effective
• •For Dose Expansion Cohort: HCC patients only: HCC confirmed by biopsy OR diagnosed by clinical and radiologic criteria; all of the following criteria must be met or a biopsy is required:
• •* Known cirrhosis or chronic hepatitis B virus (HBV) or hepatitis C virus (HCV) infection
• •* Hypervascular liver masses \> 2 cm, and either serum alpha-fetoprotein (AFP) \> 400 ng/ml
• •* AFP \> three times normal and doubling in value in the antecedent 3 months
• •* In the expansion cohort, prior treatment with sorafenib as first-line therapy allowed
• •Any number of the following prior therapies is allowed:
• •* Chemotherapy \>= 28 days prior to registration
• •* Mitomycin C/nitrosoureas \>= 42 days prior to registration
• •* Immunotherapy \>= 28 days prior to registration
• •* Biologic therapy \>= 28 days prior to registration
• •* Targeted therapy \>= 28 days prior to registration
• •* Radiation therapy \>= 28 days prior to registration
• •* Radiation to \< 25% of bone marrow
• •HCC patients only: Prior regional treatments for liver metastasis are permitted including:
• •* Selective internal radiation therapy such as brachytherapy, cyber knife, radiolabeled microsphere embolization, etc.
• •* Hepatic artery chemoembolization
• •* Hepatic artery embolization
• •* Hepatic artery infusional chemotherapy
• •* Radiofrequency ablation
• •* NOTE: patients must be \>= 4 weeks from treatment and show progressive measurable/evaluable disease in the liver after regional therapy or must have measurable disease outside the liver
• •HCC patients only: Child Pugh class A or B7 liver disease
• •Eastern Cooperative Oncology Group (ECOG) performance status =\< 2
• •Life expectancy of \> 3 months
• •Leukocytes \>= 3,000/mcL
• •Absolute neutrophil count \>= 1,500/mcL (patients may be treated with hematopoietic growth factors to achieve or maintain this level)
• •Hemoglobin \>= 9.0 g/dL
• •International normalized ratio (INR) =\< 1.4
• •Platelets \>= 100,000/mm\^3
• •Total bilirubin =\< 1.5 x ULN (institutional upper limit of normal) (patients with Gilbert's syndrome may have direct bilirubin \> 2.5 x ULN)
• •Aspartate aminotransferase (AST) (serum glutamic oxaloacetic transaminase \[SGOT\])/alanine aminotransferase (ALT) (serum glutamate pyruvate transaminase \[SGPT\]) =\< 2.5 x institutional ULN
• •Serum creatinine =\< 1.5 x ULN
• •Able to swallow and retain oral medication
• •Negative serum pregnancy test =\< 7 days prior to registration for women of childbearing potential
• •* NOTE: women will be considered not of childbearing potential if they are surgically sterile (bilateral oophorectomy or hysterectomy) and/or post-menopausal (amenorrheic for at least 12 months)
• •Ability to understand and the willingness to sign a written informed consent document
• •Willing to provide tissue samples for correlative research purposes
• •Current use of certain concomitant medications due to mechanistic-based platelet toxicities from navitoclax: clopidogrel, ibuprofen, tirofiban and other anticoagulants, drugs or herbal supplements that effect platelet function; NOTE: antiplatelet use is prohibited during the use of navitoclax; subjects who have previously received aspirin therapy for thrombosis prevention may resume a low dose (i.e., maximum 100 mg QD) of aspirin if platelet counts are stable (\>= 50,000/mm\^3) through 6 weeks of navitoclax administration; all decisions regarding treatment with aspirin therapy will be determined by the principal investigator in conjunction with the medical monitor
• •Current use of strong CYP3A inhibitors such as ketoconazole, itraconazole, voriconazole, posaconazole, nefazodone, and clarithromycin are prohibited; NOTE: moderate inhibitors of CYP3A4 should be used with caution; navitoclax is a moderate inhibitor of CYP2C8 and a strong inhibitor of CYP2C9; caution should be exercised when dosing navitoclax concurrently with CYP2C8 and CYP2C9 substrates; common CYP2C8 substrates include paclitaxel, statins and repaglinide; CYP2C9 substrates include celecoxib, phenytoin and warfarin; when possible, investigators should switch to alternative medications or monitor the patients closely
• •Concurrent use of strong CYP3A4/5 inducers such as carbamazepine, phenytoin, rifampin, and St. John's wort are prohibited
• •Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements
• •Any of the following:
• •* Pregnant women
• •* Nursing women
• •* Women of childbearing potential who are unwilling to employ adequate contraception; NOTE: should a woman become pregnant or suspect she is pregnant while she or her partner is participating in this study, she should inform her treating physician immediately; the effects of navitoclax on the developing human fetus are unknown; for this reason and because other therapeutic agents used in this trial are known to be teratogenic, women of child-bearing potential and men must agree to use adequate contraception using one of the methods listed below prior to study entry, for the duration of study participation, and up to 90 days following completion of therapy:
• •* Total abstinence from sexual intercourse (minimum one complete menstrual cycle prior to study drug administration)
• •* Vasectomized male subject or vasectomized partner of female subjects
• •* Hormonal contraceptives (oral, parenteral, transdermal or vaginal ring) for at least 3 months prior to study drug administration; if the subject is currently using a hormonal contraceptive, she should also use a barrier method during this study and for 1 month after study completion
• •* Intrauterine device (IUD)
• •* Double-barrier method: male condom plus diaphragm or vaginal cap with spermicide (contraceptive sponge, jellies or creams)
• •* Additionally, male subjects (including those who are vasectomized) whose partners are pregnant or might be pregnant must agree to use condoms for the duration of the study and for 90 days following completion of therapy
• •Human immunodeficiency virus (HIV)-positive patients on highly active antiretroviral therapy (HAART) are excluded due to possible drug-drug interactions with the investigational agent(s)
• •Underlying condition predisposing them to bleeding or currently exhibits signs of clinically significant bleeding
• •Recent history of non-chemotherapy-induced thrombocytopenic-associated bleeding =\< 1 year prior to the registration
• •History of cardiovascular disease (e.g., myocardial infraction \[MI\], thrombotic or thromboembolic event in the last 6 months)