Browsing by Author "Zhang, Yan"

Zeroth-Order (ZO) methods have been applied to solve black-box or simulation-based optimization prroblems. These problems arise in many important applications nowa- days, e.g., generating adversarial attacks on machine learning systems, learning to control the system with complicated physics structure or human in the loop. In these problem settings, the objective function to optimize does not have an explicit math- ematical form and therefore its gradient cannot be obtained. This invalidates all gradient-based optimization approaches. On the other hand, ZO methods approxi- mate the gradient by using the objective function values. Many existing ZO methods adopt the two-point feedback scheme to approximate the unknown gradient due to its low estimation variance and fast convergence speed. Specifically, two-point ZO method estimates the gradient at the current iterate of the algorithm by querying the objective function value twice at two distinct neighbor points around the current iterate. Such scheme becomes infeasible or difficult to implement when the objective function is time-varying, or when multiple agents collaboratively optimize a global objective function that depends on all agents’ decisions, because the value of the objective function can be queried only once at a single decision point. However, con- ventional ZO method based on one-point feedback is subject to large variance of the gradient estimation and therefore slows down the convergence.

In this dissertation, we propose a novel one-point ZO method based on the residual feedback. Specifically, the residual feedback scheme estimates the gradient using the residual between the values of the objective function at two consecutive iterates of the algorithm. When optimizing a deterministic Lipschitz function, we show that the query complexity of ZO with the proposed one-point residual feedback matches that of ZO with the existing two-point schemes. Moreover, the query complexity of the proposed algorithm can be improved when the objective function has Lipschitz gradient. Then, for stochastic bandit optimization problems, we show that ZO with one-point residual feedback achieves the same convergence rate as that of two-point scheme with uncontrollable data samples.

Next, we apply the proposed one-point residual-feedback gradient estimator to solve online optimizaiton problems, where the objective function varies over time. In the online setting, since each objective function can only be evaluated once at a single decision point, existing two-point ZO methods are not feasible and only one-point ZO methods can be used. We develop regret bounds for ZO with the proposed one- point residual feedback scheme for both convex and nonconvex online optimization problems. Specifically, for both deterministic and stochastic problems and for both Lipschitz and smooth objective functions, we show that using residual feedback can produce gradient estimates with much smaller variance compared to conventional one- point feedback methods. As a result, our regret bounds are much tighter compared to existing regret bounds for ZO with conventional one-point feedback, which suggests that ZO with residual feedback can better track the optimizer of online optimization problems. Additionally, our regret bounds rely on weaker assumptions than those used in conventional one-point feedback methods.

The proposed residual-feedback scheme is next decentralized to conduct dis- tributed policy optimization in the multi-agent reinforcement learning (MARL) prob- lems. Existing MARL algorithms often assume that every agent can observe the states and actions of all the other agents in the network. This can be impractical in large-scale problems, where sharing the state and action information with multi-hop neighbors may incur significant communication overhead. The advantage of the pro- posed zeroth-order policy optimization method is that it allows the agents to compute the local policy gradients needed to update their local policy functions using local estimates of the global accumulated rewards that depend on partial state and action information only and can be obtained using consensus. Specifically, the local ZO policy gradients relying on one-point residual-feedback significantly reduces the vari- ance of the local policy gradient estimates compared to the conventional one-point policy gradient estimates, improving, in this way, the learning performance. We show that the proposed distributed zeroth-order policy optimization method with constant stepsize converges to a neighborhood of the global optimal policy that depends on the number of consensus steps used to calculate the local estimates of the global accumulated rewards.Another challenge in the distributed ZO optimization problems is that the agents may conduct local updates in an asynchronous fashion when they do not have ac- cess to a global clock. To deal with this challenge, we propose an asynchronous zeroth-order distributed optimization method that relies on the proposed one-point residual feedback gradient estimator. We show that this estimator is unbiased under asynchronous updating, and theoretically analyze its convergence. We demonstrate the effectiveness of all proposed algorithms via extensive numerical experiments.

The settling of inertial particles in turbulence boundary layers plays an essential role in many meteorological, industrial and environmental processes, and is governed by multifarious mechanisms. First, turbulence alters the settling velocity of inertial particles through different effects, like preferential sweeping mechanism, loitering effect and vortex trapping. Second, the existence of a wall introduces extra effects that can influence particle settling, such as turbophoresis. The Rouse model was the most famous model in predicting particle settling in vertical wall-bounded settling. Nevertheless, it is only valid for inertia-less particles in the logarithmic region. A theory by Bragg et al., based on phase-space probability density theory, incorporates particle inertia into the Rouse model, and quantifies the contributions from the aforementioned mechanisms to the particle vertical velocity. The theory is valid for all particle Stokes numbers, yet it still lacks a closed form.In this work, one way to close the equations presented by Bragg et al. (the extended Rouse model) was examined. Using a central differencing scheme combined with an iterative method, the nonlinear second-order differential equation of the variance of vertical particle velocity was solved. The predictions of the variance of vertical particle velocity S and the particle concentration PDF ρ by the model were studied and compared to DNS. The comparison indicates that the extended Rouse model is able to predict many features of S and ρ, like the accumulation of particles close to the wall and turbophoretic drift. However, the quantitative agreement between the predictions by the model and DNS is poor. There are two probable reasons for the discrepancies between the predictions and DNS. First, the closure of the term in the equation may be a source of errors. Second, the lower boundary condition, whose validity is suspicious for particles with weak inertia, can be a reason for the discrepancies. In order to investigate the cause for the disagreement, three different boundary conditions (zero-gradient condition, asymptotic matching, iterative condition) were examined. The results indicate that the boundary conditions have a very limited influence on the predictions. As a result, the closure of the terms is more likely to be responsible for the discrepancies.

Background and Aim: Depression is a growing burden in China. There is limited national data on epidemiological studies of depression in China, as well as consistent estimates on the prevalence of depression. Therefore, we used data from two nationwide surveys to estimate prevalence of depression among middle-aged and older Chinese adults (by pre-determined subgroups). We also used multivariate analysis to explore main factors associated with depression among Chinese adults.Method: A nationally representative cross-sectional sample of 15,011 adults aged 45 to 79 years from the China Health and Retirement Longitudinal Study (CHARLS) and 8,140 adults aged 80 years and above from the Chinese Longitudinal Healthy Longevity Survey (CLHLS). Depressive symptoms were measure by the 10-item Center for Epidemiologic Studies Depression Scale (CESD-10), while probable depression was indicated by a CESD-10 score over 10. Prevalence rates of depression were estimated by age group (45-65, 65-79, 80+), sex (male, female), residence (rural, urban), and education level (no, some), incorporating sample weights. The associations between depression and marital status (married living with spouse, widowed, others including separated, unmarried), drinking status (never, former, current drinker), smoking status (never, former, current smoker), and any self-report disease (i.e., have at least one from hypertension, diabetes, cancer, lung disease, heart disease, stroke, arthritis, dyslipidemia, liver disease, kidney disease, digestive disease or not ) were examined in multivariate logistic regression and linear regression. Results: There was high prevalence of depression in middle-aged and older Chinese adults and a big difference between the two surveys, 35.0% for those aged 45-79 in CHARLS and 30.5% for those aged 80+ in CLHLS. Female, rural adults without any education had higher prevalence of depression in all age groups (45-64, 65-79, 80+). Being married but not living with spouse temporarily, separated, divorced, never married, self-reported any chronic disease were associated with higher odds of having probable depression and higher CESD-10 scores at all ages. In both age groups, being current drinker was associated with lower odds of having depression and lower CESD-10 scores. Smoking had little association with having depression in both age groups; but being a former smoker was associated with higher CESD-10 scores in CHARLS and lower scores in CLHLS. Conclusion: More attention should be devoted to the mental health of middle-aged and older Chinese adult, rural elderly women in particular. Future studies should also be aware of systematic difference when analyzing secondary data from different surveys.