In the literature, many statistical models have been used to investigate the existence of a deterministic time trend, changing persistence and nonlinearity in macroeconomic and financial data. Good understanding of these properties in a univariate time series model is crucial when making forecasts. Forecasts are used in various ways, such as helping to control risks in financial institutions and to assist in setting monetary policies in central banks. Hence, evaluating the forecast capacities of statistical models, quantifying and reducing forecast uncertainties are the main concerns of forecast practitioners. In this thesis, we propose two flexible parametric models that allow for autoregressive parameters to be time varying. One is a novel Generalised Stochastic Unit Root (GSTUR) model and the other is a Stationary Bilinear (SR) model. Bayesian inference in these two models are developed using methods on the frontier of numerical analysis. Programs, including model estimation with Markov chain Monte Carlo (MCMC), model comparison with
Bayes Factors, model forecasting and Forecast Model Averaging, are developed and made available to meet the demand of economic modelers. With an application to the S&P 500 series, we found strong evidences of a deterministic trend when we allow the persistence to change with time. By fitting the GSTUR model to monthly UK/US real exchange rate data, the Purchasing Power Parity (PPP) theory is revisited. Our findings of a changing persistence in the data suggest that the GSTUR model may reconcile the empirical findings of nonstationarity in real exchange rates with the PPP theory. The forecasting capacities of a group of nonlinear and linear models are evaluated with an application to UK inflation rates. We propose a GSTUR model to be applied with data, which contains as much information as possible, for forecasting near-term inflation rates.