Parallelize msmf corr coeff.m - Revision history

Ed at 17:34, 21 September 2020

2020-09-21T17:34:39Z

2020-09-20T17:20:08Z

2020-09-20T17:12:20Z

2018-08-03T20:48:31Z

2018-08-03T20:48:08Z

2018-07-25T20:57:34Z

Around line 120 of msmf_corr_coeff solving LP problems

2018-07-20T21:21:18Z

Around line 120 of msmf_corr_coeff solving LP problems

2018-07-20T21:19:25Z

Around line 120 of msmf_corr_coeff solving LP problems

2018-07-20T21:17:48Z

Around line 120 of msmf_corr_coeff solving LP problems

2018-07-20T21:12:45Z

Around line 120 of msmf_corr_coeff solving LP problems

@@ Line 1: / Line 1: @@
 {{Project
 |Has sponsor=McNair Center
 |Has title=Parallelize msmf corr coeff.m

@@ Line 1: / Line 1: @@
 {{Project
 |Has title=Parallelize msmf corr coeff.m
 |Has owner=Wei Wu,

@@ Line 1: / Line 1: @@
-{{McNair Projects
+{{Project
 |Has title=Parallelize msmf corr coeff.m
 |Has owner=Wei Wu,

@@ Line 74: / Line 74: @@
       end
-==Possible Further Improvements==
+=Possible Further Improvements=
 At this point I believe I have done everything I could for code performance. However if you look at the profiler, you will find that the function moments also takes a good percentage of time. I tried to parallelise moments with parfor, but it did not give any improvements.

← Older revision		Revision as of 20:48, 3 August 2018
Line 73:		Line 73:
	[asgr(:, r), ~, eflag(r)] = linprog(f(:, r), [B;-B], [constrB; constrM], [], [], zeros(L, 1), ones(L,1),options);		[asgr(:, r), ~, eflag(r)] = linprog(f(:, r), [B;-B], [constrB; constrM], [], [], zeros(L, 1), ones(L,1),options);
	end		end
		+
		+	==Possible Further Improvements==
		+	At this point I believe I have done everything I could for code performance. However if you look at the profiler, you will find that the function moments also takes a good percentage of time. I tried to parallelise moments with parfor, but it did not give any improvements.

@@ Line 47: / Line 47: @@
 ==Around line 120 of msmf_corr_coeff solving LP problems==
-For a large R, e.g. R = 200, solve 200 linear programming problems in parallel should be beneficial, especially when the each LP itself is easy to solve (takes less than 0.1 seconds each). Initially we managed to use gurobi inside a parfor:
+For a large R, e.g. R = 200, solving 200 linear programming problems in parallel should be beneficial, especially when the each LP itself is easy to solve (takes less than 0.1 seconds each). Initially we managed to use gurobi inside a parfor:
      % Gurobi inisde parfor. For large R (>100). By Wei.
       model.rhs = [full(constrB); full(constrM)];

@@ Line 67: / Line 67: @@
       end
-However using Gurobi actually did not help with the code performance. This is possibly due to the cost to create models for each Gurobi object. Gurobi is indeed the best commercial solver in the market, but it should be used for solving large LPs (hundreds of thousands of constraints), where as our Lp has only a few thousands constraints. We reversed back to using Matlab's native linprog:
+However using Gurobi actually did not help with the code performance. This is possibly due to the cost to create models for each Gurobi object. Gurobi is indeed the best commercial solver in the market, but it should be used for solving large LPs (hundreds of thousands of constraints), where as our Lp has only a few thousand constraints. We reversed back to using Matlab's native linprog:
      % Solve LPs in parallel with Matlab's native linprog
       parfor r = 1 : R

@@ Line 48: / Line 48: @@
 ==Around line 120 of msmf_corr_coeff solving LP problems==
 For a large R, e.g. R = 200, solve 200 linear programming problems in parallel should be beneficial, especially when the each LP itself is easy to solve (takes less than 0.1 seconds each). Initially we managed to use gurobi inside a parfor:
      % Gurobi inisde parfor. For large R (>100). By Wei.
       model.rhs = [full(constrB); full(constrM)];
@@ Line 61: / Line 60: @@
      % The function wrapper to use Gurobi
-    function X = solve_LP(r,f,model)
+     function X = solve_LP(r,f,model)
          model.obj = f(:,r);
          parameters.outputflag = 0;
          result = gurobi(model, parameters);
          X = result.x;
-     end
+     end

← Older revision		Revision as of 21:12, 20 July 2018
Line 48:		Line 48:
	==Around line 120 of msmf_corr_coeff solving LP problems==		==Around line 120 of msmf_corr_coeff solving LP problems==
	For a large R, e.g. R = 200, solve 200 linear programming problems in parallel should be beneficial, especially when the each LP itself is easy to solve (takes less than 0.1 seconds each). Initially we managed to use gurobi inside a parfor:		For a large R, e.g. R = 200, solve 200 linear programming problems in parallel should be beneficial, especially when the each LP itself is easy to solve (takes less than 0.1 seconds each). Initially we managed to use gurobi inside a parfor:
		+
		+	% Gurobi inisde parfor. For large R (>100). By Wei.
		+	model.rhs = [full(constrB); full(constrM)];
		+	model.A = [sparse(B);sparse(-B)];
		+	model.ub = ones(L,1);
		+	model.lb = zeros(L,1);
		+	model.sense = '<';
		+	for r = 1 : R
		+	% solve_LP is a function wrapper. It calls gurobi to solve a single LP.
		+	asgr(:,r) = solve_LP(r,f,model);
		+	end
		+
		+	% The function wrapper to use Gurobi
		+	function X = solve_LP(r,f,model)
		+	model.obj = f(:,r);
		+	parameters.outputflag = 0;
		+	result = gurobi(model, parameters);
		+	X = result.x;
		+	end