This module contains a number of utility functions with the main intent of being able to nonblockingly distribute the whole coarse problem and its rhs vectors to every thread. More...

Data Types
type	SendData
	Datatype for us with nonblocking alltoalls. More...
type	CoarseData
Functions/Subroutines
subroutine	CoarseData_Copy (cdata, cdata2)
type(SendData)	SendData_New (nproc)
subroutine	SendData_destroy (S)
subroutine	AllSendCoarselgmap (lg_cfmap, nlfc, nproc, cdisps, glg_cfmap, send)
subroutine	AllRecvCoarselgmap (send)
subroutine	AllSendCoarseMtx (A, AG, lg_cfmap, ngfc, nproc, send)
subroutine	AllRecvCoarseMtx (A, send, add)
subroutine	AllSendCoarseVector (xl, nproc, cdisps, send)
subroutine	AllRecvCoarseVector (xg, nproc, cdisps, glg_cfmap, send)
subroutine	CleanCoarse (C, R, M)
	Used for cleaning up coarse freedoms which are unused.
subroutine	setup_aggr_cdat (cdat, cdat_vec, nagrs, n, aggrnum, M)

Detailed Description

This module contains a number of utility functions with the main intent of being able to nonblockingly distribute the whole coarse problem and its rhs vectors to every thread.

The functions here are quite straightforward, except maybe the last. CleanCoarse finds freedoms which noone uses by a two phase system: First, everyone tells everyone else what nodes he doesnt need and thinks should be deleted. Second, everyone looks through the first list and if he objects to something being deleted tells the others. Only freedoms someone wanted deleted and noone objects to get deleted. The code itself is again quite straightforward.

About nonblocking allgather: the nonblockingness is achieved by moving it to another thread. That however means that

The memory it uses needs to be kept constant for longer ( so we cant change the arrays we pass to it before it is done )
No other MPI operations can be done between sends and recieves ( that includes other nonblocking gathers, which can be chained after the first one however )

Function/Subroutine Documentation

subroutine CoarseAllgathers::AllRecvCoarselgmap ( type(SendData),intent(in) send )

Parameters:

send	The sends argument output by correspondind AllSendCoarselgmap

Definition at line 167 of file CoarseAllgathers.F90.

Referenced by setup_aggr_cdat().

subroutine CoarseAllgathers::AllRecvCoarseMtx	(	type(SpMtx),intent(inout)	A,
		type(SendData),intent(in)	send,
		logical,intent(in)	add
	)

Parameters:

A	The coarse matrix - initially unusable, later global coarse matrix
send	The sends argument output by correspondind AllSendCoarseMtx

Definition at line 231 of file CoarseAllgathers.F90.

References SpMtx_arrangement::SpMtx_consolidate().

Referenced by prec2Level_exchangeMatrix().

subroutine CoarseAllgathers::AllRecvCoarseVector	(	float(kind=rk),dimension(:),intent(out)	xg,
		integer,intent(in)	nproc,
		integer,dimension(:),intent(in)	cdisps,
		integer,dimension(:),intent(in)	glg_cfmap,
		type(SendData),intent(in)	send
	)

Parameters:

xg	The global coarse vector
nproc	Number of processes
cdisps	Displacements of coarse data recieved
glg_cfmap	Coarse freemaps of other processes (assembled coarse fmap)
send	The send argument of the correspondind AllSendCoarseVector

Definition at line 292 of file CoarseAllgathers.F90.

References RealKind::rk, and globals::stream.

Referenced by prec2Level_solve().

subroutine CoarseAllgathers::AllSendCoarselgmap	(	integer,dimension(:),intent(in)	lg_cfmap,
		integer,intent(in)	nlfc,
		integer,intent(in)	nproc,
		integer,dimension(:),intent(out)	cdisps,
		integer,dimension(:),pointer	glg_cfmap,
		type(SendData),intent(out)	send
	)

Parameters:

lg_cfmap	Local-global coarse freedom map
nlfc	Number of local coarse freedoms
nproc	Number of processes
cdisps	Displacements of coarse freemaps of other nodes in acfmap
glg_cfmap	Coarse freemaps of other processes (global local-to-global crse fmap)
send	A structure for passing info to AllRecvCoarselgmap

Definition at line 120 of file CoarseAllgathers.F90.

Referenced by setup_aggr_cdat().

subroutine CoarseAllgathers::AllSendCoarseMtx	(	type(SpMtx),intent(inout)	A,
		type(SpMtx),intent(inout)	AG,
		integer,dimension(:),intent(in)	lg_cfmap,
		integer,intent(in)	ngfc,
		integer,intent(in)	nproc,
		type(SendData),intent(out)	send
	)

Parameters:

A	The coarse matrix - initially local, later unusable til AllRecv
AG	The coarse matrix - initially local, later unusable til AllRecv
lg_cfmap	Local-global coarse freedom map
ngfc	Number of global coarse freedoms
nproc	Number of processes
send	A structure for passing info to AllRecvCoarseMtx

Definition at line 175 of file CoarseAllgathers.F90.

References SpMtx_class::SpMtx_newInit().

Referenced by prec2Level_exchangeMatrix().

subroutine CoarseAllgathers::AllSendCoarseVector	(	float(kind=rk),dimension(:),intent(in)	xl,
		integer,intent(in)	nproc,
		integer,dimension(:),intent(in)	cdisps,
		type(SendData),intent(out)	send
	)

Parameters:

xl	The local coarse vector
nproc	Number of processes
cdisps	Displacements of coarse vector data
send	A variable for passing info to AllRecvCoarseVector

Definition at line 259 of file CoarseAllgathers.F90.

References not, RealKind::rk, and globals::stream.

Referenced by prec2Level_prepare().

subroutine CoarseAllgathers::CleanCoarse	(	type(CoarseGrid),intent(inout)	C,
		type(SpMtx),intent(inout)	R,
		type(Mesh),intent(in)	M
	)

Used for cleaning up coarse freedoms which are unused.

It needs to be done, since without it, singularities can and do occur in the coarse matrix.

Todo:: for current implementation which needs glg map, this could be done in that function

Parameters:

C	Coarse Grid whose structure to modify
R	The restriction matrix for finding useless freedoms
M	Fine Mesh for which to build

Definition at line 331 of file CoarseAllgathers.F90.

References SpMtx_class::D_SpMtx_ARRNG_ROWS, globals::sctls, and globals::stream.

subroutine CoarseAllgathers::CoarseData_Copy	(	type(CoarseData),intent(inout)	cdata,
		type(CoarseData),intent(in)	cdata2
	)

Definition at line 80 of file CoarseAllgathers.F90.

subroutine CoarseAllgathers::SendData_destroy ( type(SendData),intent(inout) S )

Definition at line 110 of file CoarseAllgathers.F90.

type(SendData) CoarseAllgathers::SendData_New ( integer,intent(in) nproc )

Definition at line 98 of file CoarseAllgathers.F90.

Referenced by setup_aggr_cdat().

subroutine CoarseAllgathers::setup_aggr_cdat	(	type(CoarseData),intent(out)	cdat,
		type(CoarseData),intent(out)	cdat_vec,
		integer	nagrs,
		integer,intent(in)	n,
		integer,dimension(:),pointer	aggrnum,
		type(Mesh),optional	M
	)

Definition at line 509 of file CoarseAllgathers.F90.

References AllRecvCoarselgmap(), AllSendCoarselgmap(), SpMtx_operation::exch_aggr_nums(), SpMtx_operation::exch_aggr_nums_ol0(), globals::myrank, globals::numprocs, globals::sctls, SendData_New(), and globals::stream.

CoarseAllgathers Module Reference

Data Types

Functions/Subroutines

Detailed Description

Function/Subroutine Documentation