J Cancer 2022; 13(10):3091-3102. doi:10.7150/jca.72210 This issue Cite

Research Paper

Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy

Jnaneshwari Pradeep1, Thin Thin Win2 Corresponding address, Saint Nway Aye2, Chandrashekhar T Sreeramareddy3

1. School of Medicine, International Medical University, 126, Jalan Jalil Perkasa 19, Bukit Jalil, 57000 Kuala Lumpur, Malaysia.
2. Pathology Department, School of Medicine, International Medical University, 126, Jalan Jalil Perkasa 19, Bukit Jalil, 57000 Kuala Lumpur, Malaysia.
3. Community Medicine Department, School of Medicine, International Medical University, 126, Jalan Jalil Perkasa 19, Bukit Jalil, 57000 Kuala Lumpur, Malaysia.

Citation:
Pradeep J, Win TT, Aye SN, Sreeramareddy CT. Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy. J Cancer 2022; 13(10):3091-3102. doi:10.7150/jca.72210. https://www.jcancer.org/v13p3091.htm
Other styles

File import instruction

Abstract

Graphic abstract

Background: Immune checkpoint inhibitors (ICIs) are approved as cancer immunotherapeutic agents for advanced malignant melanoma (MM) in recent years, and nivolumab and ipilimumab are the most widely used ICIs either alone or in combination. However, their efficacy and safety between single and combined ICIs are not clear. This meta-analysis (MA) is aimed to update the efficacy and safety of ICIs by comparing monotherapy and combination therapy in the treatment of advanced MM.

Method: We searched PubMed, Embase, EbscoHost and ClinicalTrials.gov for the eligible randomized controlled trials (RCTs) which compared the efficacy and safety of ICIs between a single ICI and combined ICIs. The outcomes analyzed included overall survival (OS), progression-free survival (PFS), objective response rate (ORR) and treatment-related adverse events (AEs). A fixed-effect or random-effects model was adopted depending on the study heterogeneity.

Results: A total of nine RCTs were included in this MA. Regarding the efficacy, combined nivolumab and ipilimumab therapy showed statistically significant prolonged OS and PFS with HR 0.65, 95% CI [0.53, 0.79], p <0.0001 and HR 0.48, 95% CI [0.38, 0.60], p<0.0001 respectively. Combination therapy with nivolumab and ipilimumab also showed statistically significant longer ORR than monotherapy; with RR 2.15, 95% CI [1.63, 2.84], p <0.00001. In terms of safety, the incidence of all AEs which include any AEs, high-grade, haematological, gastrointestinal, dermatological, pulmonary, liver and endocrine AEs were significantly lower with monotherapy (either nivolumab or ipilimumab) of ICI compared to combination ICI therapy with a p-value <0.00001 to 0.03.

Conclusion: Efficacy of the combined nivolumab and ipilimumab was better than a single ICI, especially in the treatment of advanced MM. Although combination therapy showed better efficacy than monotherapy, monotherapy (either nivolumab or ipilimumab) was safer than combination therapy as it tended to decrease the incidence of most of the treatment-related AEs.

Keywords: Immune checkpoint inhibitors, advanced malignant melanoma, systematic review, meta-analysis, monotherapy, combination therapy.

Introduction

Malignant melanoma (MM) is a malignant neoplasm arising from melanocytes, the melanin-producing cells of the body. It is one of the serious skin cancers and is the fifth and sixth most common cancer in males and females respectively in the United States [1]. Stage I and II MM are localized, considered early stage and curable by complete resection with a 5-year survival rate of 99.4%. However, the prognosis of regional and distant metastatic MM (stages III and IV, respectively) is generally poor, with a 5-year survival rates of 60% in stage III and 16% in stage IV [2]. The poor prognosis of advanced MM is partly due to the limited available therapeutic options [3].

In recent years, immunotherapy with immune checkpoint inhibitors (ICIs) has improved the treatment of different types of cancer including MM. Immunological checkpoint molecules suppress the attack of tumour-specific T cells, which are an integral part of anti-tumour immunity. Some ICIs block the interaction between programmed cell death-1 (PD-1) on T cells and its ligand PD-L1 on cancer cells and myeloid cells [4-6]. Other ICIs target cytotoxic T-lymphocyte antigen-4 (CTLA-4) which blocks negative signals during T-cell interaction with antigen-presenting cells and depletes regulatory T cells, thereby restoring and enhancing T-cell reactivity [7].

Novel immuno-oncologic therapies such as anti-CTLA-4 antibodies (ipilimumab) and anti-PD-1 antibodies (nivolumab and pembrolizumab) have recently achieved remarkable outcomes against advanced MM in clinical trials, with significant survival benefits and manageable safety outcomes [8-12]. ICIs can improve overall survival in both sexes with some types of advanced cancers such as MM, non-small cell lung cancer and renal cell carcinoma [13].

Nivolumab and pembrolizumab of PD-1 inhibitors, and ipilimumab of CTLA-4 inhibitors are approved by FDA in 2014 as cancer immunotherapeutic agents [14, 15]; and nivolumab and ipilimumab are the most widely used ICIs in the treatment of advanced MM, either alone or in combination [16]. The systematic review (SR) on ICIs comparing monotherapy and combination therapy on advanced solid cancers which was published in 2021 included only three randomized control studies (RCTs) for advanced MM [17]. The meta-analysis (MA) on efficacy and safety between monotherapy and combination ICIs in advanced MM was published in 2017 [18]. To fill this literature gap after the updated literature search, we aimed to perform this MA to update the efficacy and safety of ICIs comparing monotherapy and combination therapy; and to provide up-to-date comprehensive evidence for clinical decision making to choose different treatment options available in the treatment of advanced MM.

Methods

The SR and MA were performed in accordance with the updated guideline of Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) statement [19].

Identification of eligible studies

The systematic literature search was carried out in health-related electronic databases such as PubMed, Embase, EbscoHost and Cochrane library. Clinical trial studies were also searched at ClinicalTrials.gov. The search terms were immune checkpoint inhibitors, immunotherapy, ipilimumab, nivolumab, pembrolizumab and advanced malignant melanoma. The search was started in March 2021, and it was limited to the original articles published in the English language up to July 2021. To find out additional studies, reference lists of the original articles were also screened.

Inclusion and exclusion criteria

Studies in the selected articles were to meet five criteria of PICOS format: (1) Participants: individuals with histopathologically diagnosed advanced MM (Stage III and IV) of any age and sex; (2) Intervention: combined ICIs including nivolumab or pembrolizumab and ipilimumab simultaneously regardless of the dose; (3) Comparison: nivolumab or pembrolizumab or ipilimumab alone; (4) Outcomes: primary outcomes were overall survival (OS) which is defined as the time from randomization to death from any cause and treatment-related adverse effects (AEs). The secondary outcomes included progression free survival (PFS) which is defined as the time from randomization to cancer progression or death from any cause and objective response rate (ORR). (5) Studies: parallel RCTs reporting on the efficacy and or safety of ICIs; nivolumab, pembrolizumab and ipilimumab. Review articles, case reports, editorials, and studies that used other combinations were excluded for this MA.

Literature search and study selection

Two researchers (TTW, JP) conducted an independent literature search using healthcare electronic databases (PubMed, Embase and EbscoHost), Cochrane library and ClinicalKey. The articles obtained from the literature search were assessed by two researchers (TTW, JP) independently and disagreements were initially discussed. If an agreement was not reached, two researchers discussed with a third researcher (SNA) to mediate and finalize. The articles were screened according to the PRISMA flowchart. For the first stage, related articles were identified and grouped to create a total number of records. These records were screened to remove the review articles and non-relevant articles. The articles that did not meet the inclusion criteria based on abstract and title alone were excluded. Finally, full-text articles were examined to obtain the included studies required for MA.

Data extraction

Two researchers (TTW, JP) independently extracted the relevant data from the included studies using piloted data extraction sheet. Discrepancies were discussed thoroughly and finalized with a third researcher (SNA). The extracted data included study title, first author with the year of publication, country, comparison, trial phase, site of malignant melanoma, number of patients, the ratio of male to female patients, number of patients given nivolumab alone, number of patients given ipilimumab alone and number of patients given combination therapy. Outcome data included were OS, PFS, ORR, any AEs, high-grade, haematological, gastrointestinal, dermatological, pulmonary, liver and endocrine AEs.

Quality Assessment of the included studies

The Cochrane collaboration risk of bias assessment tool was used to assess the risk of bias in the included studies. This included the following assessment scopes: random sequence generation, allocation concealment, the blindness of participants and personnel, the blindness of outcome assessment, incomplete outcome data, selective reporting, and other biases. We rated each domain of the tool as 'low', 'high', or 'unclear' risk of bias at the study level and for each outcome [20].

Statistical analysis

Among the outcomes, analysis of OS and PFS was estimated as the hazard ratio (HR) and analysis of other outcomes was estimated as the risk ratios (RR) for the treatment success of monotherapy versus combination ICIs. From the statistical analysis, heterogeneities were assessed using the chi-squared test and the I² statistic. Pearson's chi-squared test was used to decide a statistically significant difference between the expected frequencies and the observed frequencies. To avoid heterogeneity, if the I² statistic was more than 50%, a random-effects model was used; and if the I² statistic was less than 50%, a fixed-effect model was used. We investigated the robustness of the review by performing sensitivity analyses when appropriate, such as performing fixed-effect models for selected outcomes and including only 'low risk of bias' outcomes, according to the summary assessment of the risk of bias. A two-tailed P value of less than 0.05 was considered statistically significant. A 95% CI was used to provide a range of values for the ORs obtained. MA was performed with Review Manager (RevMan 5.4) software.

Results

Literature search results

A total of 4036 potential studies were identified using the preliminary search strategy; 3076 were from PubMed, 554 were from EMBASE, 392 were from EbscoHost and 14 were clinical trials from ClinicalKey. A total of 325 studies were compiled after removing the duplicates. Based on the titles, 86 review articles and 123 irrelevant studies were removed. After screening abstracts of the remaining 116 studies, 102 studies were excluded as they were not specifically related to specific search criteria. Finally, 14 studies were accessed for eligibility by reading the full text. Among them, five studies were excluded with reasons based on inclusion and exclusion criteria. Therefore, the remaining nine articles were included for SR and MA. A three-phase flow chart of the study selection process based on the updated PRISMA statement 2020 is illustrated in Figure 1. A summary of the reasons for five excluded studies [21-25] is shown in Table S1. PubMed search string was (((((((immune checkpoint inhibitors) OR immunotherapy) OR ipilimumab) OR nivolumab) OR pembrolizumab) AND monotherapy) AND combination) AND melanoma.

 Table 1 

Characteristics of the included studies

Study, Year
Trial identifier (Reference)
Country Comparison Trial phaseSite of MMNo: of patientM/FN aloneI aloneCombined N + I
Amaria 2018
NCT02519322 [26]
USN+I Vs N alone2All2319/412-11
Hodi 2016 (CheckMate 069),NCT01927419 [27]France, USN+I Vs I alone2skin, mucosa14295/47-4795
Hodi 2018 (CheckMate 067),NCT01844505 [28]France, USN+I Vs I alone3skin, mucosa937605/332313311313
N+I Vs N alone
Larkin 2015NCT01844505 [29]US, Europe, Asia, AfricaN+I Vs I alone3All945610/335316315314
N+I Vs N alone
Larkin 2019 NCT01844505 [30]US, Europe, Asia, AfricaN+I Vs I alone3All937605/332313311313
N+I Vs N alone
Postow 2015 NCT01927419 [31]US, EuropeN+I Vs I alone2All14295/47-4795
Wolchok 2017 NCT01844505 [32]US, EuropeN+I Vs I alone3All945610/335316315314
N+I Vs N alone
Long 2018 NCT02374242 [33]AustraliaN+I Vs N alone2MM with brain metastasis6048/1225-35
NCT02731729 (2021) [34]USN+I Vs I alone2skin, mucosa1915/4-910

N: Nivolumab; I: Ipilimumab

 Figure 1 

PRISMA flowchart indicating study selection [19].

J Cancer Image

(View in new window)

 Figure 2A 

Assessment of risk of bias (Risk of Bias Graph).

J Cancer Image

(View in new window)

Characteristics of the included studies

The characteristics of each included study were shown in Table 1. Nine RCT studies published from 2015 to 2021 were included in this MA. Among them, eight were published in peer-reviewed journals [26-33] and one was a clinical trial that updated the results in February 2021 [34]. All these nine studies included a total of 4150 participants, comparing combined nivolumab and ipilimumab; and nivolumab or ipilimumab alone. However, combination with pembrolizumab was not reported in all the included studies. Five studies were phase 2 RCTs and 4 were phase 3 RCTs.

Assessment of risk of bias and publication bias

The results of the assessment of risk of bias are shown in the risk of bias graph (Figure 2A) and the risk of bias summary (Figure 2B). The overall risk of bias was evaluated as low risk and the quality of the studies was acceptable. Publication bias was assessed in nine included studies. Begg's and Egger's test Funnel plot showed some degree of publication bias, especially for the studies with a small sample size (Figure S1).

Efficacy of combination Vs monotherapy of ICIs

OS was reported by six included studies. Two studies [26, 30] reported combined nivolumab and ipilimumab Vs nivolumab alone and four studies [27, 28, 30, 32] reported combined nivolumab and ipilimumab Vs ipilimumab alone. OS of combination therapy was better than monotherapy with HR 0.65, 95% CI [0.53, 0.79] and it was statistically significant (p<0.0001). Subgroup analysis also showed combination therapy had significant favourable OS compared with monotherapy, nivolumab alone or ipilimumab alone with HR 0.84, 95% CI [0.71, 0.99], p 0.04 and HR 0.54, 95% CI [0.48, 0.62], p <0.00001 respectively (Figure 3A).

 Figure 2B 

Assessment of risk of bias (Risk of Bias summary).

J Cancer Image

(View in new window)

PFS was reported by seven studies. Two studies [29, 30] reported combined nivolumab and ipilimumab Vs nivolumab alone and five studies [27, 29-32] reported combined nivolumab and ipilimumab Vs ipilimumab alone. PFS of combination therapy was better than monotherapy with HR 0.48, 95% CI [0.38, 0.60] and it was statistically significant (p<0.0001). Subgroup analysis also showed combination therapy had significant favourable PFS compared with monotherapy, nivolumab alone or ipilimumab alone with HR 0.68, 95% CI [0.49, 0.94], p 0.02 and HR 0.42, 95% CI [0.37, 0.47], p <0.00001 respectively (Figure 3B).

ORR between combination therapy Vs nivolumab alone was reported by six studies [26, 28-30, 32, 33] and ORR between combination therapy Vs ipilimumab alone was reported by five studies [27-30, 32]. ORR of combination therapy was better than monotherapy with RR 2.15, 95% CI [1.63, 2.84] and it was statistically significant (p=<0.00001). Subgroup analysis also showed combination therapy had significant favourable ORR compared with monotherapy, nivolumab alone or ipilimumab alone with HR 1.32, 95% CI [1.22, 1.43], p <0.00001 and HR 3.09, 95% CI [2.74, 3.50], p <0.00001 respectively (Figure 4).

As the given data regarding PD-1/PD-L1 expression status, prior chemotherapy and dose of ICIs were not sufficiently given by primary included studies, we could not manage to perform subgroup analysis for those confounding factors.

Adverse effects of combination vs monotherapy of ICIs

In this MA, any treatment-related AEs were described by eight included studies [26-33]. High-grade AEs, gastrointestinal AEs, dermatological AEs, liver AEs and endocrine AEs were described by all nine included studies. Haematological AEs were reported by six included studies [26, 28, 30-32, 34] and pulmonary AEs were reported by seven included studies [26-28, 30-33].

Table 2 indicates the summary of the incidence of various AEs comparing monotherapy and combination therapy with RR (95% CI) and p-value. There was reduced all types of AEs with monotherapy compared with combination therapy and all AEs comparison were statistically significant with a p-value less than 0.05.

AEs of any grade between combination therapy Vs ipilimumab alone were reported by six studies [27-32] and AEs of any grade between combination therapy Vs nivolumab alone was reported by six studies [26, 28-30, 32, 33]. There were less AEs of any grade in monotherapy than combination therapy; with RR 1.07, 95% CI [1.03-1.12] and it was statistically significant (p=<0.001) (Figure 5A).

 Table 2 

Incidence of various adverse effects comparing monotherapy and combination therapy.

Adverse Effects (AEs)No. of studies RR (95% CI)  p-value  I2Statistical   method  
Any AEs81.07 [1.03-1.12]< 0.00191%Random
High grade AEs92.11 [1.71-2.59]< 0.0000187%Random
Haematological AEs61.43 [1.03-1.97]0.0361%Random
Gastrointestinal AEs91.69 [1.39-2.06]< 0.0000188%Random
Dermatological AEs91.32 [1.19-1.46]< 0.0000178%Random
Pulmonary AEs74.25[2.97-6.10]< 0.000010%Fixed
Liver AEs94.36 [3.76 -5.06]< 0.000010%Fixed
Endocrine AEs92.63 [2.16-3.21]< 0.0000152%Random
 Figure 3 

A: Forest plot for overall survival (OS); B: Forest plot for progression free survival (PFS).

J Cancer Image

(View in new window)

High-grade AEs between combination therapy Vs ipilimumab alone were reported by seven studies [27-32, 34] and high-grade AEs between combination therapy Vs nivolumab alone were reported by six studies [26, 28-30, 32, 33]. There were less high-grade AEs in monotherapy than combination therapy; with RR 2.11, 95% CI [1.71-2.59] and it was statistically significant (p=<0.00001) (Figure 5B).

Haematological AEs between combination therapy Vs ipilimumab alone were reported by four studies [28, 30-32] and combination therapy Vs nivolumab alone were reported by five studies [26, 28, 29, 32, 34]. There were reduced haematological AEs in monotherapy than combination therapy; with RR 1.43, 95% CI [1.03-1.97] and it was statistically significant (p=<0.03). However, combination therapy Vs ipilimumab alone did not show a statistically significant difference for Haematological AEs (p=0.89) (Figure S2).

Gastrointestinal AEs between combination therapy Vs ipilimumab alone were reported by six studies [27-32] and combination therapy Vs nivolumab alone were reported by seven studies [26, 28-30, 32-34]. There were reduced gastrointestinal AEs in monotherapy than combination therapy; with RR 1.69, 95% CI [1.39, 2.06] and it was statistically significant (p=<0.00001) (Figure S3).

Dermatological AEs between combination therapy Vs ipilimumab alone were reported by six studies [27-32] and combination therapy Vs nivolumab alone were reported by seven studies [26, 28-30, 32-34]. There were reduced dermatological AEs in monotherapy than combination therapy; with RR 1.32, 95% CI [1.19, 1.46] and it was statistically significant (p=<0.00001) (Figure S4).

Pulmonary AEs between combination therapy Vs ipilimumab alone were reported by five studies [28, 29, 30-32] and combination therapy Vs nivolumab alone were reported by five studies [26, 28, 30, 32, 33]. There were reduced pulmonary AEs in monotherapy than combination therapy; with RR 4.25, 95% CI [2.97, 6.10] and it was statistically significant (p=<0.00001) (Figure S5).

Liver AEs between combination therapy Vs ipilimumab alone were reported by six studies [27-32] and combination therapy Vs nivolumab alone were reported by six studies [26, 28-30, 32, 33]. There were reduced liver AEs in monotherapy than combination therapy; with RR 4.36, 95% CI [3.76, 5.06] and it was statistically significant (p=<0.00001) (Figure S6).

Endocrine AEs between combination therapy Vs ipilimumab alone were reported by six studies [27-32] and combination therapy Vs nivolumab alone were reported by seven studies [26, 28-30, 32-34]. There were reduced endocrine AEs in monotherapy than combination therapy; with RR 2.63, 95% CI [2.16, 3.21] and it was statistically significant (p=<0.00001) (Figure S7).

Discussion

In recent years, the role of immunotherapy in cancer treatment has expanded and now it is the first choice of treatment intervention in many cancers [35]. Both in-vivo and clinical studies of combined chemotherapy and ICIs showed promising results in many solid tumours such as renal cell carcinoma and non-small cell lung carcinoma [36, 37]. Immunotherapeutic drugs are used as single ICI or combined ICIs or combined with chemotherapeutic drugs. A Cochrane systematic review on systematic treatment for metastatic cutaneous MM reported that PD-1 inhibitors (nivolumab and pembrolizumab) improved OS and PFS compared to chemotherapy and CTLA-4 inhibitors (ipilimumab and tremelimumab) [38].

In the treatment of many solid cancers such as non-small cell lung cancers, oesophageal carcinoma and malignant mesothelioma, the efficacy of combined ICIs is better than single ICI [39]. Combined two ICIs are the most promising approaches in the treatment of advanced MM [40]. The majority of combination ICIs used are CTLA-4 inhibitors (ipilimumab) and PD-1 inhibitors (nivolumab or pembrolizumab). These drugs have been authorized by the United States Food and Drug Administration (FDA) for the treatment of advanced MM in 2011 and 2014 [41-43].

 Figure 4 

Forest plot for objective response rate (ORR).

J Cancer Image

(View in new window)

 Figure 5 

A: Forest plot for adverse effects of any grade; B: Forest plot for high grade adverse effects.

J Cancer Image

(View in new window)

In this MA, we only compared the combination therapy of nivolumab and ipilimumab with either nivolumab or ipilimumab alone as included primary studies reported this combination only. OS showed that there was a statistically significant difference between combined nivolumab and ipilimumab with either nivolumab or ipilimumab alone. Therefore, OS of combination ICIs therapy was better than monotherapy. It also showed a clear statistically significant difference in the PFS between combined nivolumab and ipilimumab with either nivolumab or ipilimumab alone. Therefore, PFS of combination ICIs therapy was better than monotherapy. ORR of combination ICIs therapy was also better than that of monotherapy significantly. These results on OS and PFS are concordant with the results of a Cochrane systematic review which studied the outcomes of combined ICIs in the treatment of advanced non-small cell lung cancer. However, combined ICIs did not improve PFS and ORR compared to platinum-based chemotherapy in that study [44].

Our MA showed that combination therapy improved OS, PFS and ORR compared to nivolumab alone or ipilimumab alone, and this finding is concordant with the results of the other two MAs on the efficacy of ICIs in various types of cancers [17, 39]. Another MA on the efficacy of combined ICIs also reported that improved ORR was seen in the combined nivolumab and ipilimumab treatment group in advanced MM [18]. A phase 2 RCT that studied the effects of combined nivolumab and ipilimumab in advanced MM metastasized to the brain also reported clinically meaningful efficacy with combined ICIs therapy [45]. Among the targeted therapies of metastatic MM, combination ICIs had clinically significant intracranial efficacy [46]. It was also reported that improved ORR and better response rate were seen with combined nivolumab and ipilimumab therapy in various types of solid cancers including MM [39]. This might be due to nivolumab which is PD-1 inhibitor, and it was reported that anti-PD1 monoclonal antibodies are better than anti‐CTLA4 monoclonal antibodies in terms of OS in the treatment of metastatic cutaneous MM [38]. Another MA on ICIs in metastatic mucosal MM also reported that monoclonal antibodies targeting the PD-1 and PD-L1 interaction seemed to be more effective than targeting CTLA-4 in the treatment of MM [47]. In the treatment of many solid tumours, combined antibodies of PD-1 and PD-L1 have been widely used and it provides better results than anti-CTLA4 therapy [48].

In this MA, we could not manage to analyze complete and partial responses as some of the included studies reported only ORR without specific data on complete and partial response rates. Some included studies reported OS and PFS based on the expression of PD-1 and PD-L1 by the tumour cells. As not all included studies did not report those expressions, analyses based on those expressions were not included in this MA. Among nine included studies, two studies reported that both combination ICIs therapy and nivolumab monotherapy showed improved OS, ORR and PFS regardless of PD-L1 expression [29, 32]. A MA of ICIs on non-small cell carcinoma reported that combined ICIs prolonged OS compared to platinum-based chemotherapy in people with PD-L1 expression ≥50% [44].

Many combination ICIs have been developed in the treatment of solid tumours, especially MM, and nivolumab plus ipilimumab is the most used combination [49]. However, nivolumab is a PD-1 inhibitor and ipilimumab is a CTLA-4 inhibitor, their mechanisms of action are not the same and not complementary to each other [50]. It is assumed that antitumor activity of CTLA-4 inhibitors may be enhanced by tumour-specific T effector cells suppression through the PD-L1/PD-1 pathway and that immune suppression is partially mediated by CTLA-4 inhibitors itself [51]. The efficacy of PD-1 inhibitors might be compromised by the lack of full activation of tumour-specific effector T cells mediated by CTLA-4, which is also thought to be aggravated by upregulation of CTLA-4 induced by the PD-1 inhibition itself [50, 52]. Therefore, the higher immune-mediated anti-tumour activity gave better efficacy with nivolumab plus ipilimumab than nivolumab or ipilimumab alone in the treatment of advanced MM and non-small cell lung carcinoma [53]. The success of cancer treatment with ICIs is not only due to targeting cancer cell destruction through the activation of the host immune system, but it also targets the cancer-immune environment with activation of tumour infiltrating T lymphocytes [54].

Another ICI used in the treatment of MM is relatilimab which block the lymphocyte-activation gene 3 (LAG-3) expressed on immune cells including T cells, and negatively regulates T-cell proliferation and effector T-cell function. A clinical trial (NCT03470922) has been done on efficacy of combine relatilimab and nivolumab [55], and it showed better PFS although it showed no new safety signals [56]. As there is only one clinical trial on it, we did not include the efficacy and safety regarding combination with relatilimab in this MA.

ICIs can cause immune-related AEs as the mechanism of ICI action relies on the inhibition of the physiological brake of immune activation and they often have off-target effects resulting in immune-mediated inflammation of diverse organs or tissues [54]. The spectrum of AEs caused by ICIs is different from those of chemotherapy and they are mainly autoimmune and autoinflammatory complications [57]. CTLA-4 inhibitors carry a higher risk and severity of immune-related AEs compared to other ICIs [58]. The potential pathophysiological mechanisms involved in the development of immune-related AEs are T cell-mediated mechanism, B cell-mediated effects, CTLA-4 expression in the tissue and inflammatory cytokine-mediated mechanism [59].

In this MA, we analyzed common treatment-related AEs caused by a single ICI either nivolumab or ipilimumab alone and combined nivolumab plus ipilimumab. The incidence of AEs analyzed in this MA were any treatment-related AEs, high-grade AEs, haematological, gastrointestinal, dermatological, pulmonary, liver and endocrine AEs. It showed that a single ICI either nivolumab or ipilimumab alone reduced all types of AEs compared to combined nivolumab plus ipilimumab, and it showed statistically significant results for all AEs. Therefore, monotherapy either nivolumab or ipilimumab alone is safer than combination ICIs therapy. This finding is concordant with the result reported by a MA that studied potential immune-related AEs in monotherapy and combination therapy of ipilimumab, nivolumab, and pembrolizumab in advanced MM [60]. Another MA that studied the safety of combination ICIs therapy in advanced solid tumours also reported that severe AEs were associated with combination ICIs compared with monotherapy [17]. A MA of ICIs in non-small cell lung cancer which compared ICIs and platinum-based chemotherapy reported that high-grade AEs were rare with single ICI compared to chemotherapy; however, the frequency of those AEs was not significantly different between combination ICIs and chemotherapy [44]. In the treatment of metastatic cutaneous MM, although anti-PD1 monoclonal antibodies are the least toxic regimen, combination ICIs increase toxicity compared to chemotherapy and other targeted therapies such as BRAF inhibitors [38].

Most of the studies in our MA reported grade 3 and 4 AEs as high-grade AEs. No grade 5 AEs resulting in death were reported in all included studies. The most common haematological AEs were anaemia, leucopenia and thrombocytopenia. The most common gastrointestinal AEs were vomiting, diarrhoea and colitis. The most common dermatological AEs reported were skin rash and pruritis. Pulmonary AEs were reported as pneumonia or pneumonitis. Liver AEs included increased serum levels of alanine transaminases and aspartate transaminases. Endocrine AEs included hyperthyroidism, hypothyroidism and hypophysitis/ hypopituitarism, and only a few reported hyperglycaemia/ diabetes mellitus. Most of the treatment-related AEs developed within a month of the last dose in nivolumab plus ipilimumab group and the majority were resolved as they were not severe and manageable [61]. A MA that studied different doses of nivolumab and ipilimumab in the combination group found that AEs were slightly increased in the group with nivolumab 1 mg/kg plus ipilimumab 3 mg/kg compared with nivolumab 3 mg/kg plus ipilimumab 1 mg/kg [39]. In our MA, many include studies did not specifically mention the dose of ICIs.

CTLA-4 inhibitor (ipilimumab) was mostly associated with an increased incidence of gastrointestinal, renal, dermatological, and endocrine AEs. The AEs of PD-1 inhibitors (nivolumab or pembrolizumab) were the endocrine, gastrointestinal, liver, musculoskeletal, nervous system, renal, and respiratory AEs [60]. Cessation of the ICIs, initiation of steroids and supportive therapy were the preferred choice of treatment for those AEs [62, 63]. A MA showed toxicity-related fatality rates of 0.36% (anti-PD-1), 0.38% (anti-PD-L1), 1.08% (anti-CTLA-4), and 1.23% (PD-1/PD-L1 plus CTLA-4). Although rare, ICI-related deaths may occur when severe iatrogenic AEs such as myocarditis, encephalitis, or acute hypophysitis are not readily diagnosed and these AEs were treated with high dose steroids and more potent immunosuppressors [64]. Most of the included studies in our MA reported that most treatment-related AEs were generally manageable, and no study reported death from AEs. Various biomarkers are known to be associated with the onset of immune-related AEs caused by ICIs. Although most of these biomarkers such as T cell, B cells, microbiome and genomic biomarkers are not bona fide predictive markers, some of them have potential clinical utility [65]. The autoimmune status of the patient should be checked before choosing ICI for the treatment.

This MA had few limitations in conducting data extraction and MA of those data. For efficacy and safety, different included studies used different follow-up duration. If there was a standardized follow-up duration, data and results especially on efficacy will be more accurate. Another limitation was that our MA was based on unadjusted data analysis, and we could not manage to analyze based on other confounding factors such as age, sex, PD-1/PD-L1 expression, prior chemotherapy and dose of ICIs etc. Another limitation was that the studies on combination with relatlimab, anti- LAG-3 were not included in this MA as enough number of primary studies to perform MA have not been published.

In conclusion, this MA showed that the efficacy of the combined ICIs was more favourable than single ICI in terms of OS, PFS and ORR. There was a significant difference in the combination Vs monotherapy in terms of OS, PFS and ORR. Although combination ICIs therapy showed better efficacy than monotherapy, monotherapy (either nivolumab or ipilimumab) was safer than combination therapy as it tended to decrease the incidence of most of the treatment-related AEs. Taking the patient's safety as our main priority, the most appropriate clinical practice would be prescribing monotherapy over combination therapy in the treatment of advanced MM. Starting with monotherapy using a single ICI followed by adding another ICI can be considered to minimize the AEs and improve efficacy in the treatment of advanced MM. Studies on how immune-related AEs can be separated from the anti-tumour effects of ICIs and the identification of specific biomarkers to predict the development of the toxicities need to be thoroughly undertaken for the improvement of patient outcomes.

Competing Interests

The authors have declared that no competing interest exists.

Supplementary Material

Supplementary figures.

Attachment

Acknowledgements

The authors are grateful to the participants and researchers of the primary studies in this meta-analysis; and to the International Medical University (IMU) Malaysia for allowing us to perform this study [ID: BMS I/2020(17)].

This research work of systematic review has been registered on PROSPERO (Registration number: CRD42021193237).

Author's information: ORCID ID: Thin Thin Win - https://orcid.org/0000-0002-6468-5761

Ethics approval

This research work has been approved by IMU Joint-Committee on Research & Ethics with project ID: BMS I/2020(17).

Funding

This research work was supported by a Grant from the International Medical University, Kuala Lumpur, Malaysia (Project ID: BMS I/2020(17).

Author contributions

Research design: TTW, SNA; Data collection and extraction: TTW, JP, SNA; Statistical analysis: TTW, JP, CTS; Drafting manuscript: JP, TTW, SNA, CTS; Final approval of manuscript: JP, TTW, SNA, CTS.

References

1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2020. CA Cancer J Clin. 2020;70:7-30

2. National Cancer Institute SEER Program. Cancer stat facts: melanoma of the skin. Accessed May 2. 2021 https://seer.cancer.gov/statfacts/html/melan.html

3. Maverakis E, Cornelius LA, Bowen GM, Phan T, Patel FB, Fitzmaurice S. et al. Metastatic melanoma:a review of current and future treatment options. Acta Derm Venereol. 2015;95:516-524

4. Elias R, Morales J, Rehman Y, Khurshid H. Immune checkpoint inhibitors in older adults. Curr Oncol Rep. 2016;18:47

5. Elias R, Karantanos T, Sira E, Hartshorn KL. Immunotherapy comes of age: immune aging & checkpoint inhibitors. J Geriatr Oncol. 2017;8:229-35

6. Nishijima TF, Muss HB, Shachar SS, Moschos SJ. Comparison of efficacy of immune checkpoint inhibitors (ICIs) between younger and older patients: a systematic review and meta-analysis. Cancer Treat Rev. 2016;45:30-7

7. Romano E, Kusio-Kobialka M, Foukas PG, Baumgaertner P, Meyer C, Ballabeni P. et al. Ipilimumab dependent cell-mediated cytotoxicity of regulatory T cells ex vivo by nonclassical monocytes in melanoma patients. Proc Natl Acad Sci USA. 2015;112:6140-5

8. Hodi FS, O'Day SJ, McDermott DF, Weber RW, Sosman JA, Haanen JB. et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010;363:711-723 37

9. Robert C, Thomas L, Bondarenko I, O'Day S, Weber J, Garbe C. et al. Ipilimumab plus dacarbazine for previously untreated metastatic melanoma. N Engl J Med. 2011;364:2517-2526 38

10. Schadendorf D, Hodi FS, Robert C, Weber JS, Margolin K, Hamid O. et al. Pooled analysis of longterm survival data from phase II and phase III trials of ipilimumab in unresectable or metastatic melanoma. J Clin Oncol. 2015;33:1889-94

11. Weber JS, D'Angelo SP, Minor D, Hodi FS, Gutzmer R, Neyns B. et al. Nivolumab versus chemotherapy in patients with advanced melanoma who progressed after antiCTLA-4 treatment (CheckMate 037): A randomised, controlled, openlabel, phase 3 trial. Lancet Oncol. 2015;16:375-384 41

12. Robert C, Schachter J, Long GV, Arance A, Grob JJ, Mortier L. et al. Pembrolizumab versus ipilimumab in advanced melanoma. N Engl J Med. 2015;372:2521-32

13. Conforti F, Pala L, Bagnardi V, De Pas T, Martinetti M, Viale G. et al. Cancer immunotherapy efficacy and patients' sex: a systematic review and meta-analysis. Lancet Oncol. 2018;19:737-746

14. Leite KR, Reis ST, Junior JP, Zerati M, Gomes Dde O, Camara-Lopes LH. et al. PD-L1 expression in renal cell carcinoma clear cell type is related to unfavorable prognosis. Diagn Pathol. 2015;10:189

15. Patnaik A, Kang SP, Rasco D, Papadopoulos KP, Elassaiss-Schaap J, Beeram M. et al. Phase I study of pembrolizumab (MK-3475; anti-PD-1 monoclonal antibody) in patients with advanced solid tumors. Clini Cancer Res. 2015;21:4286-93

16. Amaral T, Kiecker F, Schaefer S, Stege H, Kaehler K, Terheyden P. et al. Combined immunotherapy with nivolumab and ipilimumab with and without local therapy in patients with melanoma brain metastasis: a DeCOG* study in 380 patients. J Immunother Cancer. 2020;8(1):e000333

17. Ma X, Zhang Y, Wang S, Wei H, Yu J. Immune checkpoint inhibitor (ICI) combination therapy compared to monotherapy in advanced solid cancer: A systematic review. J Cancer. 2021;12:1318-1333

18. Hao C, Tian J, Liu H, Li F, Niu H, Zhu B. Efficacy and safety of anti-PD-1 and anti-PD-1 combined with anti-CTLA-4 immunotherapy to advanced melanoma: A systematic review and meta-analysis of randomized controlled trials. Medicine (Baltimore). 2017;96:e7325

19. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD. et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71

20. Higgins JP, Churchill R, Chandler J, Cumpston MS, editor(s), Cochrane Handbook for Systematic Reviews of Interventions Version 5.2.0 (updated June 2017), Cochrane, 2017

21. Schadendorf D, Wolchok JD, Hodi FS, Chiarion-Sileni V, Gonzalez R, Rutkowski P. et al. Efficacy and safety outcomes in patients with advanced melanoma who discontinued treatment with nivolumab and ipilimumab because of adverse events: A pooled analysis of randomized phase II and III trials. J Clin Oncol. 2017;35:3807-3814

22. Rozeman EA, Menzies AM, van Akkooi ACJ, Adhikari C, Bierman C, van de Wiel BA. et al. Identification of the optimal combination dosing schedule of neoadjuvant ipilimumab plus nivolumab in macroscopic stage III melanoma (OpACIN-neo): a multicentre, phase 2, randomised, controlled trial. Lancet Oncol. 2019;20:948-960

23. Lebbé C, Meyer N, Mortier L, Marquez-Rodas I, Robert C, Rutkowski P. et al. Evaluation of two dosing regimens for nivolumab in combination with ipilimumab in patients with advanced melanoma: results from the phase IIIb/IV CheckMate 511 trial. J Clin Oncol. 2019;37:867-875

24. NCT01844505. Phase 3 study of nivolumab or nivolumab plus ipilimumab versus ipilimumab alone in previously untreated advanced melanoma. (CheckMate 067). Accessed April 20. 2021 https://clinicaltrials.gov/ct2/show/NCT01844505

25. NCT01927419. Study of nivolumab (BMS-936558) plus ipilimumab compared with ipilimumab alone in the treatment of previously untreated, unresectable, or metastatic Melanoma (CheckMate 069). Accessed April 20. 2021 https://clinicaltrials.gov/ct2/show/NCT01927419

26. Amaria RN, Reddy SM, Tawbi HA, Davies MA, Ross MI, Glitza IC. et al. Neoadjuvant immune checkpoint blockade in high-risk resectable melanoma. Nat Med. 2018;24:1649-1654

27. Hodi FS, Chesney J, Pavlick AC, Robert C, Grossmann KF, McDermott DF. et al. Combined nivolumab and ipilimumab versus ipilimumab alone in patients with advanced melanoma: 2-year overall survival outcomes in a multicentre, randomised, controlled, phase 2 trial. Lancet Oncol. 2016;17:1558-1568

28. Hodi FS, Chiarion-Sileni V, Gonzalez R, Grob JJ, Rutkowski P, Cowey CL. et al. Nivolumab plus ipilimumab or nivolumab alone versus ipilimumab alone in advanced melanoma (CheckMate 067): 4-year outcomes of a multicentre, randomised, phase 3 trial. Lancet Oncol. 2018;19:1480-1492

29. Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Cowey CL, Lao CD. et al. Combined Nivolumab and Ipilimumab or Monotherapy in Untreated Melanoma. N Engl J Med. 2015;373:23-34

30. Larkin J, Chiarion-Sileni V, Gonzalez R, Grob JJ, Rutkowski P, Lao CD. et al. Five-Year Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma. N Engl J Med. 2019;381:1535-1546

31. Postow MA, Chesney J, Pavlick AC, Robert C, Grossmann K, McDermott D. et al. Nivolumab and ipilimumab versus ipilimumab in untreated melanoma. N Engl J Med. 2015;372:2006-17

32. Wolchok JD, Chiarion-Sileni V, Gonzalez R, Rutkowski P, Grob JJ, Cowey CL. et al. Overall Survival with Combined Nivolumab and Ipilimumab in Advanced Melanoma. N Engl J Med. 2017;377:1345-1356

33. Long GV, Atkinson V, Lo S, Sandhu S, Guminski AD, Brown MP. et al. Combination nivolumab and ipilimumab or nivolumab alone in melanoma brain metastases: a multicentre randomised phase 2 study. Lancet Oncol. 2018;19:672-681

34. NCT02731729. Ipilimumab vs Ipilimumab plus Nivolumab in patients with stage III-IV melanoma who have progressed or relapsed on PD-1 inhibitor therapy. Accessed June 2. 2021 https://clinicaltrials.gov/ct2/show/NCT02731729

35. Kooshkaki O, Derakhshani A, Hosseinkhani N, Torabi M, Safaei S, Brunetti O. et al. Combination of Ipilimumab and Nivolumab in Cancers: From Clinical Practice to Ongoing Clinical Trials. Int J Mol Sci. 2020;21:4427

36. Cui S. Immunogenic chemotherapy sensitizes renal cancer to immune checkpoint blockade therapy in preclinical models. Med Sci Monit. 2017;23:3360-3366

37. Papadimitrakopoulou V, Gadgeel SM, Borghaei H, Gandhi L, Patnaik A, Powell SF. et al. First-line carboplatin and pemetrexed (CP) with or without pembrolizumab (pembro) for advanced nonsquamous NSCLC: updated results of KEYNOTE-021 cohort G. J Clin Oncol. 2017;35(Suppl 15):9094

38. Pasquali S, Hadjinicolaou AV, Chiarion Sileni V, Rossi CR, Mocellin S. Systemic treatments for metastatic cutaneous melanoma. Cochrane Database Syst Rev. 2018 6;2(2): CD011123

39. Chen J, Li S, Ya Q, Du N, Fu X, Lou Y. et al. The efficacy and safety of combined immune checkpoint inhibitors (nivolumab plus ipilimumab): a systematic review and meta-analysis. World J Surg Onc. 2020 18, 150

40. Schadendorf D, Hodi FS, Robert C, Weber JS, Margolin K, Hamid O. et al. Pooled analysis of longterm survival data from phase II and phase III trials of ipilimumab in unresectable or metastatic melanoma. J Clin Oncol. 2015;33:1889-1894 39

41. Traynor K. Ipilimumab approved for metastatic melanoma. Am J Health Syst Pharm. 2011;68:768

42. Raedler LA. Keytruda (Pembrolizumab): first PD-1 inhibitor approved fornpreviously treated unresectable or metastatic melanoma. Am Health Drug Benefits. 2015;8:96-100

43. Raedler LA. Opdivo (Nivolumab): second PD-1 inhibitor receives FDA approval for unresectable or metastatic melanoma. Am Health Drug Benefits. 2015;8:180-3

44. Ferrara R, Imbimbo M, Malouf R, Paget-Bailly S, Calais F, Marchal C. et al. Single or combined immune checkpoint inhibitors compared to first-line platinum-based chemotherapy with or without bevacizumab for people with advanced non-small cell lung cancer. Cochrane Database Syst Rev. 2020;12(12):CD013257

45. Tawbi HA, Forsyth PA, Algazi A, Hamid O, Hodi FS, Moschos SJ. et al. Combined Nivolumab and Ipilimumab in Melanoma Metastatic to the Brain. N Engl J Med. 2018;379:722-730

46. Zbytek B, Carlson JA, Granese J, Ross J, Mihm MC Jr, Slominski A. Current concepts of metastasis in melanoma. Expert Rev. Derm. 2008 3, 569-585

47. Li J, Kan H, Zhao L, Sun Z, Bai C. Immune checkpoint inhibitors in advanced or metastatic mucosal melanoma: a systematic review. Ther Adv Med Oncol. 2020;12:1758835920922028

48. Vafaei S, Zekiy AO, Khanamir RA, Zaman BA, Ghayourvahdat A, Azimizonuzi H. et al. Combination therapy with immune checkpoint inhibitors (ICIs); a new frontier. Cancer Cell Int. 2022;22:2

49. Fujimura T, Kambayashi Y, Sato Y, Tanita K, Amagai R, Hashimoto A. et al. Successful treatment of unresectable advanced melanoma by administration of nivolumab with ipilimumab before primary tumor resection. Front Med. 2019;6:140

50. Ott PA, Hodi FS, Robert C. CTLA-4 and PD-1/PD-L1 blockade: new immunotherapeutic modalities with durable clinical benefit in melanoma patients. Clin Cancer Res. 2013;19:5300-9

51. Pardoll DM. The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer. 2012; 12(4): 252-264. Pardoll DM. The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer. 2012;12:252-264

52. Curran MA, Montalvo W, Yagita H, Allison JP. PD-1 and CTLA-4 combination blockade expands infiltrating T cells and reduces regulatory T and myeloid cells within B16 melanoma tumors. Proc Natl Acad Sci U S A. 2010;107:4275-80

53. Buchbinder EI, Desai A. CTLA-4 and PD-1 pathways: similarities, differences, and implications of their inhibition. Am J Clin Oncol. 2016;39:98-106

54. Robert C. A decade of immune-checkpoint inhibitors in cancer therapy. Nat Commun. 2020;11:3801

55. NCT03470922. A study of relatlimab plus nivolumab versus nivolumab alone in participants with advanced melanoma (RELATIVITY-047). Accessed May 12. 2022 https://www.clinicaltrials.gov/ct2/show/NCT03470922

56. Tawbi HA, Schadendorf D, Lipson EJ, Ascierto PA, Matamala L, Castillo Gutiérrez E, et al; RELATIVITY-047 Investigators. Relatlimab and nivolumab versus nivolumab in untreated advanced melanoma. N Engl J Med. 2022;386:24-34

57. Martins F, Sofiya L, Sykiotis GP, Lamine F, Maillard M, Fraga M. et al. Adverse effects of immune-checkpoint inhibitors: epidemiology, management and surveillance. Nat Rev Clin Oncol. 2019;16:563-80

58. Michot JM, Bigenwald C, Champiat S, Collins M, Carbonnel F, Postel-Vinay S. et al. Immune-related adverse events with immune blockade: a comprehensive review. Eur J Cancer. 2016;54:139-48

59. König D, Läubli H. Mechanisms of Immune-Related Complications in Cancer Patients Treated with Immune Checkpoint Inhibitors. Pharmacology. 2021;106:123-136

60. Almutairi AR, McBride A, Slack M, Erstad BL, Abraham I. Potential immune-related adverse events associated with monotherapy and combination therapy of ipilimumab, nivolumab, and pembrolizumab for advanced melanoma: A systematic review and meta-analysis. Front Oncol. 2020;10:91

61. Wen X, Wang Y, Ding Y, Li D, Li J, Guo Y. et al. Safety of immune checkpoint inhibitors in Chinese patients with melanoma. Melanoma Res. 2016;26:284-9

62. Bajwa R, Cheema A, Khan T, Amirpour A, Paul A, Chaughtai S. et al. Adverse effects of immune checkpoint inhibitors (programmed death-1 inhibitors and cytotoxic T-lymphocyte associated protein-4 inhibitors): results of a retrospective study. J Clin Med Res. 2019;11:225-36

63. Pollack MH, Betof A, Dearden H, Rapazzo K, Valentine I, Brohl AS. et al. Safety of resuming anti-PD-1 in patients with immune-related adverse events (irAEs) during combined anti-CTLA-4 and anti-PD1 in metastatic melanoma. Ann Oncol. 2018;29:250-5

64. Wang DY, Salem JE, Cohen JV, Chandra S, Menzer C, Ye F. et al. Fatal toxic effects associated with immune checkpoint inhibitors: a systematic review and meta-analysis. JAMA Oncol. 2018;4:1721-1728

65. Sullivan RJ, Weber JS. Immune-related toxicities of checkpoint inhibitors: mechanisms and mitigation strategies. Nat Rev Drug Discov. 2022;21:495-508

Author contact

Corresponding address Corresponding author: Dr Thin Thin Win (Email: thinthinwinedu.my)


Received 2022-2-20
Accepted 2022-7-4
Published 2022-8-8


Citation styles

APA
Pradeep, J., Win, T.T., Aye, S.N., Sreeramareddy, C.T. (2022). Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy. Journal of Cancer, 13(10), 3091-3102. https://doi.org/10.7150/jca.72210.

ACS
Pradeep, J.; Win, T.T.; Aye, S.N.; Sreeramareddy, C.T. Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy. J. Cancer 2022, 13 (10), 3091-3102. DOI: 10.7150/jca.72210.

NLM
Pradeep J, Win TT, Aye SN, Sreeramareddy CT. Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy. J Cancer 2022; 13(10):3091-3102. doi:10.7150/jca.72210. https://www.jcancer.org/v13p3091.htm

CSE
Pradeep J, Win TT, Aye SN, Sreeramareddy CT. 2022. Efficacy and Safety of Immune Checkpoint Inhibitors for Advanced Malignant Melanoma: A Meta-Analysis on Monotherapy Vs Combination Therapy. J Cancer. 13(10):3091-3102.

This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/). See http://ivyspring.com/terms for full terms and conditions.