-
Ensuring Fair LLM Serving Amid Diverse Applications
Authors:
Redwan Ibne Seraj Khan,
Kunal Jain,
Haiying Shen,
Ankur Mallick,
Anjaly Parayil,
Anoop Kulkarni,
Steve Kofsky,
Pankhuri Choudhary,
Renèe St. Amant,
Rujia Wang,
Yue Cheng,
Ali R. Butt,
Victor Rühle,
Chetan Bansal,
Saravan Rajmohan
Abstract:
In a multi-tenant large language model (LLM) serving platform hosting diverse applications, some users may submit an excessive number of requests, causing the service to become unavailable to other users and creating unfairness. Existing fairness approaches do not account for variations in token lengths across applications and multiple LLM calls, making them unsuitable for such platforms. To addre…
▽ More
In a multi-tenant large language model (LLM) serving platform hosting diverse applications, some users may submit an excessive number of requests, causing the service to become unavailable to other users and creating unfairness. Existing fairness approaches do not account for variations in token lengths across applications and multiple LLM calls, making them unsuitable for such platforms. To address the fairness challenge, this paper analyzes millions of requests from thousands of users on MS CoPilot, a real-world multi-tenant LLM platform hosted by Microsoft. Our analysis confirms the inadequacy of existing methods and guides the development of FairServe, a system that ensures fair LLM access across diverse applications. FairServe proposes application-characteristic aware request throttling coupled with a weighted service counter based scheduling technique to curb abusive behavior and ensure fairness. Our experimental results on real-world traces demonstrate FairServe's superior performance compared to the state-of-the-art method in ensuring fairness. We are actively working on deploying our system in production, expecting to benefit millions of customers world-wide.
△ Less
Submitted 24 November, 2024;
originally announced November 2024.
-
Gallium Oxide Heterojunction Diodes for Improved High-Temperature Performance
Authors:
Shahadat H. Sohel,
Ramchandra Kotecha,
Imran S Khan,
Karen N. Heinselman,
Sreekant Narumanchi,
M Brooks Tellekamp,
Andriy Zakutayev
Abstract:
$β$-Ga${_2}$O${_3}…
▽ More
$β$-Ga${_2}$O${_3}$ based semiconductor devices are expected to have significantly improved high-power and high-temperature performance due to its ultra-wide bandgap of close to 5 eV. However, the high-temperature operation of these ultra-wide-bandgap devices is usually limited by the relatively low 1-2 eV built-in potential at the Schottky barrier with most high-work-function metals. Here, we report heterojunction p-NiO/n-$β$-Ga${_2}$O${_3}$ diodes fabrication and optimization for high-temperature device applications, demonstrating a current rectification ratio of more than 10${^6}$ at 410°C. The NiO heterojunction diode can achieve higher turn-on voltage and lower reverse leakage current compared to the Ni-based Schottky diode fabricated on the same single crystal $β$-Ga${_2}$O${_3}$ substrate, despite charge transport dominated by interfacial recombination. Electrical characterization and device modeling show that these advantages are due to a higher built-in potential and additional band offset. These results suggest that heterojunction p-n diodes based on $β$-Ga${_2}$O${_3}$ can significantly improve high-temperature electronic device and sensor performance.
△ Less
Submitted 31 March, 2022;
originally announced April 2022.
-
Stakeholders interdependencies and their role in sustainable business model innovation
Authors:
Iqra Sadaf Khan,
Jukka Majava
Abstract:
Sustainable innovation requires in-time development, diversification and transformation of business models from one to another. Business model innovation, development and transformation for sustainability incorporates economic, environmental and social value by advancing the management of the stakeholders into the business model. Except a little research on business model inter-dependencies, scant…
▽ More
Sustainable innovation requires in-time development, diversification and transformation of business models from one to another. Business model innovation, development and transformation for sustainability incorporates economic, environmental and social value by advancing the management of the stakeholders into the business model. Except a little research on business model inter-dependencies, scant research has been done on stakeholders inter-dependencies in order to understand their nature and relationship while developing or transforming a business model and creating an impact on environment, society and economy. Therefore, current research uses actor dependency model to analyze four different kind of inter-dependencies, namely, goal-dependency, task-dependency, resource-dependency and soft-goal dependency. The ecology of business model experimentation map is used as a tool for practical understanding of sustainable business modelling with a multi-actor approach in a workshop setting. The findings will help to understand how stakeholders depend on each other while developing a business model for sustainability and innovation.
△ Less
Submitted 30 March, 2021;
originally announced March 2021.
-
$Mg_xZn_{1-x}O$ contact to $CuGa_3Se_5$ absorber for photovoltaic and photoelectrochemical devices
Authors:
Imran S. Khan,
Christopher P. Muzzillo,
Craig L. Perkins,
Andrew G. Norman,
James Young,
Nicolas Gaillard,
Andriy Zakutayev
Abstract:
$CuGa_3Se_5$ is a promising candidate material with wide band gap for top cells in tandem photovoltaic (PV) and photoelectrochemical (PEC) devices. However, traditional CdS contact layers used with other chalcopyrite absorbers are not suitable for $CuGa_3Se_5$ due to the higher position of its conduction band minimum. $Mg_xZn_{1-x}O…
▽ More
$CuGa_3Se_5$ is a promising candidate material with wide band gap for top cells in tandem photovoltaic (PV) and photoelectrochemical (PEC) devices. However, traditional CdS contact layers used with other chalcopyrite absorbers are not suitable for $CuGa_3Se_5$ due to the higher position of its conduction band minimum. $Mg_xZn_{1-x}O$ is a transparent oxide with adjustable band gap and conduction band position as a function of magnesium composition, but its direct application is hindered by $CuGa_3Se_5$ surface oxidation. Here, $Mg_xZn_{1-x}O$ is investigated as a contact (n-type buffer or window) material to $CuGa_3Se_5$ absorbers pretreated in $Cd^{2+}$ solution, and an onset potential close to 1 V vs RHE in 10 mM hexaammineruthenium (III) chloride electrolyte is demonstrated. The $Cd^{2+}$ surface treatment changes the chemical composition and electronic structure of the $CuGa_3Se_5$ surface, as demonstrated by photoelectron spectroscopy measurements. The performance of $CuGa_3Se_5$ absorber with $Cd^{2+}$ treated surface in the solid-state test structure depends on the Zn/Mg ratio in the $Mg_xZn_{1-x}O$ layer. The measured open circuit voltage close to 1 V is promising for tandem PEC water splitting with $CuGa_3Se_5$/$Mg_xZn_{1-x}O$ top cells.
△ Less
Submitted 4 October, 2020;
originally announced October 2020.
-
Inverse Z-spectrum analysis for MT- and spillover-corrected and T1-compensated steady-state pulsed CEST-MRI - application to pH-weighted MRI of acute stroke
Authors:
Moritz Zaiss,
Junzhong Xu,
Steffen Goerke,
Imad S. Khan,
Robert J. Singer,
John C. Gore,
Daniel F. Gochberg,
Peter Bachert
Abstract:
Endogenous chemical exchange saturation transfer (CEST) effects are always diluted by competing effects such as direct water proton saturation (spillover) and macromolecular magnetization transfer (MT). This leads to T2-and MT-shine-through effects in the actual biochemical contrast of CEST. Therefore, a simple evaluation algorithm which corrects the CEST signal was searched for. By employing a re…
▽ More
Endogenous chemical exchange saturation transfer (CEST) effects are always diluted by competing effects such as direct water proton saturation (spillover) and macromolecular magnetization transfer (MT). This leads to T2-and MT-shine-through effects in the actual biochemical contrast of CEST. Therefore, a simple evaluation algorithm which corrects the CEST signal was searched for. By employing a recent eigenspace theory valid for spinlock and continuous wave (cw) CEST we predict that the inverse Z-spectrum is beneficial to Z-spectrum itself. Based on this we propose a new spillover- and MT-corrected magnetization transfer ratio (MTRRex) yielding Rex, the exchange dependent relaxation rate in the rotating frame. For verification, the amine proton exchange of creatine in solutions with different agar concentration was studied experimentally at clinical field strength of 3T. In contrast to the compared standard evaluation for pulsed CEST experiments, MTRasym, our approach shows no T2 or MT shine through effect. We demonstrate that spillover can be corrected properly and also quantitative evaluation of pH and creatine concentration is possible which proves MTRRex as quantitative CEST-MRI method. A spillover correction is of special interest for clinical static field strengths and protons resonating near the water peak. This is the case for -OH-CEST effects like gagCEST or glucoCEST, but also amine exchange of creatine or glutamate which require high B1. Although, only showed for amine exchange, we propose our normalization to work generally for DIACEST, PARACEST in slow- and fast exchange regime not just as a correction, but also for quantitative CEST-MRI. Applied to acute stroke induced in rat brain, the corrected CEST signal shows significantly higher contrast between stroke area and normal tissue as well as less B1 dependency compared to conventional approaches.
△ Less
Submitted 26 September, 2013; v1 submitted 26 February, 2013;
originally announced February 2013.
